Operate updated 2026-05-22

Audit export

Hash-chained audit log on the Mastio: view in the dashboard, verify the chain online, export to NDJSON for offline forensic review, and re-verify the chain with the standalone CLI.

Audit export

Who this is for: a compliance operator, security reviewer, or investigator who needs a tamper-evident record of what happened on the Mastio for a given time window. Every significant event lands in one of two append-only, SHA-256 hash-chained tables:

  • audit_log — the primary chain: every gateway action (LLM chat egress, MCP tool calls, admin mutations, token mints/revokes). One global chain per Mastio. Rows written since migration 0042 use the v2 canonical, which binds the agent’s DPoP key thumbprint (dpop_jkt) and the on-behalf-of user into the hash. The RFC 3161 and Merkle anchors are built over this chain.
  • local_audit — the per-org chain: enrollment, session open, A2A messaging, key rotation, framework update apply.

What’s in the chains

The audit_log table:

ColumnMeaning
idAuto-increment primary key
timestampISO 8601 UTC
agent_idActor
actione.g. egress_llm_chat, tool_call, api_token.mint
tool_nameTool when applicable
statussuccess / denied / error
detailEvent-specific payload
request_id, duration_msCorrelation + timing
chain_seqMonotonic global sequence (UNIQUE)
prev_hashLink to the prior row’s row_hash (genesis for the first)
row_hashSHA-256 over the canonical representation of the row
hash_formatv2 (binds the two columns below) or NULL/v1 legacy
dpop_jktRFC 9449 key thumbprint bound into v2 hashes
on_behalf_of_user_idADR-032 attribution bound into v2 hashes

The local_audit table:

ColumnMeaning
idAuto-increment primary key
timestampISO 8601 UTC
event_typee.g. agent.enrolled, mcp.tool_call, admin.update_applied, pki.rotate_ca
agent_idActor — agent URI when applicable
session_idSession correlation when applicable
org_idOrg scope
detailsEvent-specific JSON payload
resultok / denied / error
entry_hashSHA-256 over canonical representation of the row
previous_hashLink to the prior row’s entry_hash (chain pointer)
chain_seqMonotonic sequence within the chain
peer_org_id, peer_row_hashReserved for future cross-org reconciliation. Empty on standalone deploys.

The local_audit chain is per-org; the standalone Mastio (the default bundle deploy) writes a single per-org chain. The audit_log chain is global for the whole Mastio.

Tamper-evidence properties (both tables):

  • Append-only via DB triggers — even admin DB access cannot UPDATE / DELETE without dropping the triggers (an act that itself leaves audit trail).
  • Each row’s hash (row_hash / entry_hash) covers a canonical serialization including the previous row’s hash, so any modification of an earlier row breaks every subsequent hash.

1. View in the dashboard

The Mastio dashboard ships a unified audit log viewer:

https://mastio.example.com/proxy/audit

It merges two streams:

  • Legacy admin events (auth, enrollment, agent CRUD, policy) from the older audit_log table.
  • Hash-chained local_audit rows (MCP tool calls, A2A messages, sessions, key rotations).

Filters at the top let you narrow by event_type, agent_id, session_id, and time window. Each row is expandable to inspect the details JSON.

2. Verify chain integrity online

The dashboard exposes a Verify chain button that hits POST /proxy/audit/verify and renders the result inline:

ADMIN_SECRET="$(grep ^MCP_PROXY_ADMIN_SECRET proxy.env | cut -d= -f2)"
CSRF="$(...)"  # extract from dashboard cookie / session

curl -sk -X POST "https://localhost:9443/proxy/audit/verify" \
     -H "Cookie: session=$DASHBOARD_SESSION" \
     -H "X-CSRF-Token: $CSRF"

The endpoint walks every local_audit row in chain_seq order, recomputes entry_hash, and confirms previous_hash links match. Response shape:

{
  "ok": true,
  "entries": 8421,
  "agents": 12,
  "orgs": 1
}

On failure:

{
  "ok": false,
  "failure": {
    "expected": "3f4a...2b1e",
    "observed": "9e0c...a55d",
    "scope": "agent-chain",
    "id": 4217
  }
}

A failure means the chain was tampered with after write (UPDATE / DELETE bypass, or storage corruption). Treat as a security incident: capture the full local_audit table to cold storage, identify the row range, then escalate.

3. Export for offline forensic review

GET /v1/admin/audit/export streams a verifier-ready NDJSON bundle — both chains plus the TSA / Merkle anchor rows — with admin-secret auth:

ADMIN_SECRET="$(grep ^MCP_PROXY_ADMIN_SECRET proxy.env | cut -d= -f2)"

# Everything: audit_log + local_audit + anchors
curl -sk -H "X-Admin-Secret: ${ADMIN_SECRET}" \
     "https://mastio.example.com:9443/v1/admin/audit/export?chain=both" \
     -o bundle.ndjson

# Only the primary chain, a chain_seq window
curl -sk -H "X-Admin-Secret: ${ADMIN_SECRET}" \
     "https://mastio.example.com:9443/v1/admin/audit/export?chain=audit_log&since_seq=5000&until_seq=9000" \
     -o window.ndjson

# Only one org's local_audit chain
curl -sk -H "X-Admin-Secret: ${ADMIN_SECRET}" \
     "https://mastio.example.com:9443/v1/admin/audit/export?chain=local_audit&org_id=acme" \
     -o acme.ndjson

Parameters: chain=audit_log|local_audit|both (default both); org_id filters local_audit rows only (the audit_log chain is global — combining it with chain=audit_log is a 400); since_seq / until_seq are inclusive chain_seq bounds (a window that doesn’t start at the genesis is forward-verified only — the verifier says so, qualifies its verdict, and refuses such bundles under --require-genesis); include_anchors=false drops the anchor rows. Seq windows are meant for audit_log: a local_audit window that starts past an org’s first row will fail that org’s chain walk — export local_audit org-complete instead. The response streams in keyset-paginated batches, so a multi-100k-row export doesn’t hold DB locks for the duration of the download.

Every entry row carries an explicit "chain" key so the verifier never guesses the schema.

SQL (alternative)

For a window keyed on timestamp rather than chain_seq, or where the admin secret isn’t available to the exporting party, query the database directly.

SQLite (default bundle)

docker compose -p cullis-mastio exec mcp-proxy \
    sqlite3 /data/mcp_proxy.db \
    -cmd '.mode json' \
    "SELECT * FROM local_audit
     WHERE timestamp >= '2026-04-01T00:00:00Z'
       AND timestamp <  '2026-05-01T00:00:00Z'
     ORDER BY chain_seq;" \
    | python3 -c 'import sys, json; [print(json.dumps(r)) for r in json.load(sys.stdin)]' \
    > audit-april-2026.ndjson

The same recipe works for the primary chain — substitute audit_log and ORDER BY chain_seq; the verifier auto-detects the schema per row.

Postgres (opt-in)

psql -h "$PG_HOST" -U cullis -d cullis -A -t -c \
    "COPY (SELECT row_to_json(t) FROM (
       SELECT * FROM local_audit
       WHERE timestamp >= '2026-04-01T00:00:00Z'
         AND timestamp <  '2026-05-01T00:00:00Z'
       ORDER BY chain_seq) t) TO STDOUT;" \
    > audit-april-2026.ndjson

Store the resulting file in append-only cold storage (S3 Object Lock, WORM filesystem). The NDJSON is self-contained — no Mastio access needed for re-verification years later.

4. Verify offline with the standalone CLI

The repo ships scripts/cullis-audit-verify.py — a zero-network, offline verifier consumable by anyone holding the NDJSON dump (your compliance team, an external auditor, a federated peer reconciling cross-org events).

python scripts/cullis-audit-verify.py --bundle bundle.ndjson

The verifier walks both chains: the global audit_log chain (recomputing every row_hash from content, including the v2 DPoP/on-behalf-of binding) and each per-org local_audit chain. --chain audit_log|local_audit forces the schema interpretation for the paranoid auditor (default auto: the export’s explicit chain key wins, with a field-shape heuristic for SQL dumps). TSA anchor rows are matched against the audit_log chain they were minted over. For dispute-grade verification add --require-anchors and --tsa-trust-store.

Example output:

Bundle: audit-april-2026.ndjson
Per-org chains: 1
  acme: 8421 entries, chain_seq 1 → 8421, head 9e0c...a55d
✓ Per-org chain walk: entry_hash + previous_hash verified for all 8421 rows
✓ Legacy global chain (chain_seq IS NULL rows): no violations
✓ No TSA anchor rows present in this window

cullis-audit-verify.py exits with:

  • 0 — all checks passed
  • 2 — chain tamper detected (mismatch or break, on either chain)
  • 3 — TSA anchor mismatch (when applicable; the bundle includes RFC 3161 / mock TSA tokens and one doesn’t match its row)
  • 4 — reserved (cross-org reconciliation, future use)
  • 5 — unrecognized TSA token format
  • 8--require-anchors set but zero dispute-grade anchors in the bundle
  • 9 — bundle schema unrecognized (a row matches neither chain shape, or contradicts an explicit --chain override)

Troubleshoot

Verify returns ok: false immediately : Someone wrote to local_audit with the triggers dropped, or the DB was restored from a backup older than the current chain head and a few rows got duplicated. Capture the full table for forensics, then run the verifier with --bundle on a SQL dump to identify the exact tamper window.

Chain-integrity verify is slow on large deploys : The verify endpoint walks the whole chain. For deploys with millions of rows, run it during a maintenance window or rely on the offline verifier against a recent NDJSON dump. A streaming / incremental verifier is on the roadmap.

Verifier reports “unrecognized TSA format” : The chain contains TSA anchor rows in a format the verifier doesn’t know how to validate (e.g. a new RFC 3161 variant or a custom anchor). Update cullis-audit-verify.py to the latest from scripts/ and retry. TSA anchors only land in chains produced by deploys with TSA wiring enabled — standalone bundle Mastios don’t generate them by default.

Export contains rows with entry_hash=NULL : Those are pre-chain rows from before hash chaining was enabled. The verifier accepts them as legacy and skips them. Modern installs (Mastio v0.5+) write every row with a chain entry from boot.

Next

  • Disaster recovery — back up the chain as part of the regular bundle backup
  • Rotate keys — rotate events land in the chain as pki.rotate_ca / agent.cert_rotated
  • Apply updates — framework update events land as admin.update_applied / admin.update_rolled_back
  • Runbook § monitoring — Prometheus counters mirror the most common audit event types for live dashboards