# Airlock Spec Design Guide Airlock specs work best when they separate submitted business facts from Airlock control-plane state. ## Payload vs Airlock State A file payload should contain the facts supplied by the person, agent, source system, or business process. Airlock should carry the lifecycle around that payload: file identity, path, filename, workflow state, audit events, attachments, expectations, retention, and authorization. For a submitted reimbursement, good payload fields are: ```text employee_username, manager_username, receipt_date, merchant, amount, currency, category, business_purpose ``` The receipt belongs in Airlock attachments. The file identity belongs in the manifest path and filename. The approval journey belongs in Airlock workflow and events. Avoid asking a user or agent to invent fields such as: ```text reimbursement_id, approval_status, approved_by, approved_at, workflow_status, workflow_step, processing_context ``` Those values are either Airlock metadata, reviewer output, or downstream system state. They should not be required in the initial submitter payload unless they come from a real upstream system and are part of the business fact being captured. ## Status Fields Are Not Banned Status can be legitimate business data. For example, a Shopify product status, a Bill.com payment status, or a source-system subscription status may be a real source fact. The rule is narrower: do not duplicate Airlock lifecycle state in the primary payload. If the field means "where this file is in Airlock review," use Airlock workflow. If it means "what the source system reported," model it explicitly and name the source. ## Submit-To-Workflow Pattern Some business workflows should skip a visible "extra click" for the submitter. For example, an agent may prepare a reimbursement for asmith and land it in the Submitted state so a reviewer can act immediately. Keep that as Airlock workflow, not payload shape: ```text validate business payload -> load file -> advance workflow to Submitted ``` An MCP tool or agent skill can present this as one action, such as "submit this reimbursement", but the secure implementation is still `load_data` followed by an authorized `edit_file_workflow` transition. If the reviewer has a problem, they return the file to Draft with a workflow comment. The principal then sees the returned Draft item in My Work, fixes or resends the content, and resubmits before the expectation due date. Do not add `submitted_at`, `approval_status`, `workflow_step`, or reviewer notes to the source file just to support this journey. ## Agent Architecture Patterns Airlock works best when agents are modeled as explicit participants, not hidden shortcuts around human permissions. Prefer these patterns: - Assign agents their own Airlock roles. Do not reuse a human's role assignment just because the agent works for that human. - Use delegation when an agent should act for a specific person. A delegated call should keep both identities visible: the actor agent and the principal user. Delegation is not impersonation. - Treat most production agents as tied to one human or owner account, similar to a dedicated assistant. Group-owned agents are possible, but they need clearer operating rules and stronger audit expectations. - Keep workflow as the communication channel for review, pushback, and handoff. A file can land in Submitted so reviewers can act immediately, while Draft remains useful as the pushback state when reviewers need changes. - Prefer polling Airlock work lists, materialized spec tables, or reference specs for near-term observer agents. Snowflake notifications may become useful for real-time signaling, but polling the governed Airlock surface is simpler and easier to audit today. - Use materialized specs plus read-only reference specs for high-read streams. The demo `posts` spec writes governed records and materializes `AIRLOCK_DATA.ACTIVE.T_POSTS`; `published_posts` exposes that table back to users and agents as read-only reference data. - Chain agents through governed outputs, not private memory. A reviewer or observer can observe Approved budget requests and then begin a downstream budget-package process, or observe an accepted application and begin a final invitation process. ## Read-Only Observer Pattern Use `observer_access` when an Airlock role should monitor a spec without reading governed payload rows or gaining mutation rights: ```json { "observer_access": { "enabled": true, "observer_roles": [ { "role_name": "finance_observer", "path_filter": "finance", "workflow_states": ["Approved"] } ] } } ``` Assign each observer agent its own Snowflake username and Airlock role. Grant that principal the Snowflake application role `app_observer`, then assign the Airlock role normally. The application role limits the principal to read-only `observe.*` procedures; the Airlock assignment and spec rule limit which specs, paths, and workflow states are visible. Observe-only procedure calls do not consume a named Airlock seat. Use the locked `global_observer` Airlock role only for account-wide compliance or audit principals. It cannot own specs, receive guest or reviewer grants, or manage child roles. Do not use one shared integration identity for multiple channels or agents: separate identities preserve attribution and make each channel's observation scope independently revocable. ## Restricted Reference Pattern Some references should not support broad paginated reads, even when the underlying Snowflake object is already projected, masked, or hashed safely. For example, a support agent may have a legitimate case-specific reason to see one customer identity record, but no reason to retrieve many customer records at once. Use `restricted_reference` for that interaction shape: ```json { "core_config": {"is_reference_spec": true}, "restricted_reference": { "object_key": "customers", "lookup_key": "customer_id", "requires_purpose": true, "purpose_pattern": "support_case:*", "disable_general_reads": true }, "reference_config": { "object_paths": [ { "object_key": "customers", "database_name": "APP_DATA", "schema_name": "SAFE", "object_name": "CUSTOMER_IDENTITY_SUPPORT", "object_kind": "view", "readable_columns": ["customer_id", "ssn_last4"] } ] } } ``` Airlock does not define the mask, hash, token, or projection for sensitive values. Snowflake policies, secure views, or upstream preparation own that value shape. Airlock owns the governed interface: - `airlock.agent.select_reference_data` refuses broad reads for a restricted reference. - `airlock.agent.get_reference_record` returns at most one configured lookup result. - `action_limit` Expectations can cap aggregate usage before any restricted result is returned. - `observe.reference_context`, `observe.usage_limits`, `observe.usage_limit`, `observe.explain_access`, and `observe.governance_map` expose the policy and counted activity without querying raw reference rows. Demo coverage: - `make demo-posts` creates the `posts` write spec and `published_posts` reference spec. - `make demo-posts-discovery` verifies a human user can discover, validate, and read the posts flow through agent procedures. - `make demo-posts-agent-discovery` verifies csmith, acting as an `agent` role, can validate, load, and read posts without admin procedures. - `make demo-posts-json` verifies csmith can author a post as records JSON, adapt it to the existing `AGENT.VALIDATE_DATA` / `AGENT.LOAD_DATA` file-content contract, and read it back through `published_posts`. - `make demo-okf-bundle` verifies an OKF-style Markdown knowledge bundle can be validated locally, loaded through `ADMIN.LOAD_OKF_BUNDLE`, and read through the accepted concept metadata view. - `make demo-delegation-discovery` verifies csmith can discover the received reimbursement delegation and validate on behalf of asmith while remaining isolated from asmith's direct workflow work list. ## Governed Knowledge Bundles Records JSON is for submitted tabular business facts. OKF-style knowledge bundles are different: they are Markdown files with YAML frontmatter, stable paths, links, and source citations. Treat them as governed knowledge packages, not as CSV rows or attachments pretending to be records. The Airlock OKF profile is documented in `docs/development/airlock_okf_profile_rfc.md`. The local helper validates a directory or zip archive before Snowflake work: ```python from airlock.okf_knowledge_bundle import validate_okf_bundle validation = validate_okf_bundle("sales_context_bundle.zip") assert validation.ok request = validation.metadata_sync_request( spec_name="okf_knowledge", path="default", filename="sales_context", ) ``` The raw bundle file is still governed by Airlock file storage and workflow. For specs that declare `core_config.payload_adapter = "okf_knowledge_bundle"`, `airlock.admin.load_okf_bundle(...)` writes the bundle bytes, creates the manifest row, evaluates PDP and Expectations, and projects parsed concept rows into `core.OKF_CONCEPT_METADATA` in one governed call. `airlock.admin.sync_okf_bundle_metadata(...)` remains available when a bundle file already has an active manifest row and only the parsed projection needs to be refreshed. Agents should read accepted/current concept metadata from `AIRLOCK_DATA.ACTIVE.V_OKF_CONCEPT_METADATA` and cite bundle version plus concept path. The profile supports OKF v0.2 `sources`, `generated`, `verified`, lifecycle, and computation descriptors, with legacy v0.1 timestamp/source-link support. These are producer claims, not Airlock permissions or approval. Bundle loading never executes computations or attesters. Keep bundles Markdown-only; the existing size, path, link, workflow, and Expectation safeguards still apply. ## Canonical Records And File Adapters For agents and generated apps, the natural Airlock input shape is JSON records: an array of objects whose keys match the spec `column_config`. CSV and Excel are separate file adapters into that record model. They remain useful for human uploads and compatibility with existing `load_data` / `validate_data` procedures, but they are not the business concept Airlock is trying to model. Good agent-facing shape: ```json { "spec_name": "posts", "filename": "post_001", "records": [ { "post_id": "POST-001", "reply_to_post_id": "", "submitted_by": "csmith", "body": "Please make reimbursements easier", "tags": "#request #reimbursements", "related_area": "finance", "related_process": "employee reimbursements", "details": { "source": {"channel": "agent"}, "request": {"desired_outcome": "an agent can prepare the upload"} } } ] } ``` Adapter rules: - Describe the spec first and order fields by `column_config`. - Reject fields that are not declared by the spec. - Validate JSON values against declared Airlock column types before adapter conversion. Use JSON-native booleans and numbers; use ISO strings for date, datetime, and UUID values; use real objects/arrays for declared `variant` columns. The local helper also treats `object`, `array`, and `json` as variant-style aliases for tooling and future contracts; current Airlock specs should prefer the explicit `variant` type. - Validate declared `variant_shape` rules before adapter conversion when a record provides structured data. `variant_shape` rules must target declared variant-style columns. A bad nested value should fail with the original record, field, and JSON path, not later as an opaque CSV/load issue. - Serialize nested objects only into `variant` columns. - Preserve nulls and empty values so Airlock validation can report required field issues consistently. - Use the records JSON -> CSV bridge only for specs that can be represented as one flat row per record. Nested structures need declared `variant` columns or a purpose-built future adapter. - Convert CSV and Excel uploads into the same record model where practical, but keep CSV/Excel-specific parsing and validation separate from native records JSON validation. - In current tooling, `airlock.records_json.validate_records_payload` returns local adapter issues and `airlock.records_json.records_payload_to_load_data_args` produces the procedure-ready `spec_name`, `filename`, and CSV `file_content` for existing `AGENT.VALIDATE_DATA` / `AGENT.LOAD_DATA` procedures. `airlock.records_json.records_payload_to_load_data_request` can additionally merge separate load context into a structured `AGENT.LOAD_DATA` request object for agent and generated-app adapters. Local helper checks cover payload shape, declared fields, simple `not_null`, `unique`, and `allowed_values` constraints inside the submitted records, declared column types, and declared `variant_shape` rules. The helper can also apply a `max_inline_bytes` ceiling so generated apps fail locally before sending oversized inline CSV to Snowflake. The installed app still enforces access, expectations, workflow, attachments, references, audit, and final validation. - `make demo-posts-json` is the end-to-end smoke for this pattern: csmith's connection builds a unique records JSON post, converts it locally, calls only `AGENT.*` procedures, and reads the accepted row back through `published_posts`. - If a records payload names a `spec_name`, compare it to the described `core_config.spec_name` before conversion. A mismatch means the agent or generated app is using the wrong contract. - Keep procedure context outside the payload. `path_scope`, `in_app_role`, `on_behalf_of_user`, `delegation_id`, workflow target, and attachment content are explicit Airlock call/tool context, not fields inside `records`. - Use stable filenames or record keys for retry-safe generated apps. Retrying one logical submission should not silently create a new filename unless the user explicitly wants a new record batch. This is a semantic input contract, not a reason to bypass Airlock. PDP, expectations, workflow, attachments, audit logging, retention, reference checks, and final validation still happen through installed Airlock procedures. ## VARIANT Columns Use `variant` for flexible business or provider details that are real evidence: raw provider payloads, sparse optional attributes, nested review evidence, and fields that are still evolving. Do not use `variant` as a junk drawer for Airlock process state. If a field is workflow, approval routing, UI state, debug output, or "for office use only" metadata, put it in Airlock workflow/events/expectations or a separate review or commitment spec. When a nested field becomes operationally important, promote it deliberately: ```text raw VARIANT payload -> variant_shape rule -> promoted path -> explicit column ``` Keep payloads bounded. Prefer one business event, source record, review result, or provider response per row. If an upstream JSON document is an outer array of records, model it as multiple rows or use a light adapter step rather than one giant `variant`. ## Review And Commitment Specs Derived specs are allowed to carry derived work products. A review spec can record check results, evidence, recommendations, and references to the file it reviewed. A payment commitment spec can record the outbound packet that was prepared from approved upstream files. Even there, prefer Airlock file identity for upstream references: ```text reimbursement_spec_name, reimbursement_path, reimbursement_filename ``` Do not require a submitter-created id just so downstream records can link back. If the upstream system already has a stable id, capture it. If not, Airlock's manifest identity is the stable operational handle. ## Agent Rule Agents should always call `describe_spec` before building a file. They should send records whose keys are declared in `column_config`, satisfy file rules and attachments, and avoid inventing ids, approval fields, workflow fields, or extra metadata columns. If a tool accepts JSON records, it may convert them to CSV for the current stored procedures, but Airlock procedure results remain the source of truth. After a successful load, agents should report the returned `PATH`, `FILENAME`, `STATUS`, `ISSUES`, and workflow state.