Authoring certification and release evidence

This page documents how Databricks resource management capabilities are tested, certified against a real workspace, and promoted to stable. It is aimed at release engineers and operators evaluating the stability claims; day-to-day authoring users do not need it.

Test coverage gate

The release gate runs pnpm test:coverage:authoring. It covers approval binding and replay, principal/input/call mismatches, grant expiry and persistence parsing, governance extraction and denial lineage, Jobs policy/wire contracts, Cloudflare and Temporal grant propagation, and the certification lifecycle runner. Genie contract tests additionally cover deterministic version-2 serialization, validation, every management/ACL wire path, ETag/fingerprint conflicts, durable ownership, managed-only deletion, conversation terminal failures, response bounds, cancellation, history, comments, and attachment-specific results. The current focused gate requires at least 85% lines/functions/statements and 80% branches; the implementation currently exceeds those thresholds. Package suites add every authoring client variant, bundle fail-closed checks, durable stores, CLI build/deploy, and documentation examples.

The Databricks package additionally runs a native-SDK boundary suite. It executes generated clients over an instrumented HTTP transport to assert official request paths, JSON field serialization, credential headers, OBO headers, oneof handling, bigint identifiers, and SDK error mapping. A source allowlist quarantines preview-only endpoints, every @databricks/sdk-* dependency must share one exact reviewed version, and test:sdk-runtime imports the production bundle under Node 22. These contract checks run without workspace credentials; the lifecycle gate below proves actual service behavior.

Protected workspace lifecycles

Yes, protected CI can create real workspace resources and remove them. Enable the destructive gate only in an isolated certification workspace:

export FABRIC_DATABRICKS_AUTHORING_TEST=1
export DATABRICKS_OBO_TOKEN=...
export DATABRICKS_AUTHORING_AUTH_MODE=on-behalf-of
export DATABRICKS_AUTHORING_EXECUTING_PRINCIPAL_KIND=user
export DATABRICKS_AUTHORING_EXECUTING_PRINCIPAL_ID=...
export DATABRICKS_AUTHORING_EXECUTING_UC_PRINCIPAL=user@example.com
export DATABRICKS_AUTHORING_STEWARD_ID=data-platform-certifier
export DATABRICKS_AUTHORING_JOB_SPEC='{"name":"jobs","tasks":[{"taskKey":"probe","notebook":{"notebookPath":"/Shared/probe"}}]}'
export DATABRICKS_AUTHORING_COMPUTE_POLICY='{}'
export DATABRICKS_AUTHORING_GENIE_SPEC='{"version":2,"title":"genie","parentPath":"/Shared","warehouseId":"warehouse-id","dataSources":[{"table":"main.cert.fixture"}]}'
export DATABRICKS_AUTHORING_GENIE_QUESTION='How many rows are available?'
export FABRIC_DATABRICKS_REQUIRED_CHECKS='identity,approval-provenance,jobs-authoring,lakeflow-authoring,ai-search-admin,serving-admin,uc-admin-obo-execution,workspace-write,secrets-write,genie-authoring'
# Also configure the Lakeflow, AI Search, serving, UC, workspace, and secret fixtures below.
node packages/databricks/dist/certify.js

The remaining fixture variables are DATABRICKS_AUTHORING_LAKEFLOW_SPEC, DATABRICKS_AUTHORING_AI_SEARCH_SPEC, DATABRICKS_AUTHORING_SERVING_SPEC, DATABRICKS_AUTHORING_GENIE_SPEC, DATABRICKS_AUTHORING_UC_CATALOG, DATABRICKS_AUTHORING_UC_SCHEMA_PREFIX, DATABRICKS_AUTHORING_WORKSPACE_ROOT, DATABRICKS_AUTHORING_SECRET_SCOPE_PREFIX, and the protected secret DATABRICKS_AUTHORING_SECRET_VALUE. Every generated name carries the reserved fabric-harness-authoring-cert- prefix (or its Unity Catalog-safe underscore form). The separate namespace prevents the authoring sweeper from touching long-lived baseline certification fixtures. Each check executes create, verify, mutate, verify, delete, and verify-delete through approval-bound governed tools. Failures trigger reverse-order cleanup; a scheduled sweeper removes crash leftovers and fails if deletion leaks.

The Genie lifecycle uses the same approval-bound ToolDefs as production, asks a real Conversation API question, verifies that the created Agent's ACL is readable through the exact-pinned official Access Management SDK, accepts Databricks' successful soft-delete (has been trashed) as deletion evidence, clears durable Harness ownership when a retry finds that the managed Agent was already trashed, and fails if the Agent remains in the active inventory. Agent Mode uses the separate optional DATABRICKS_GENIE_AGENT_MODE_ID check and is not implied by management certification.

To promote Agent Mode itself, also configure DATABRICKS_GENIE_AGENT_MODE_QUESTION and optionally DATABRICKS_GENIE_AGENT_MODE_TIMEOUT_MS, then dispatch with require_agent_mode: true. The workflow adds genie-agent-mode to the required checks and fails on missing configuration, malformed or unordered SSE, missing output, a missing terminal event, or a provider-side Beta denial.

Classic compute has an independent promotion gate as well. Configure DATABRICKS_AUTHORING_CLASSIC_JOB_SPEC with a newCluster task bound to an allowed policyId, then dispatch with certification_scope: classic-compute in a protected classic-workspace environment. Certification requires only identity, approval provenance, the classic Job fixture, and its compute policy in that scope; it creates, reads, mutates, and deletes the Job definition through the governed tool path and does not start the cluster. The same DATABRICKS_AUTHORING_COMPUTE_POLICY allowlists its policy, runtime, node type, workers, and tags. Serverless workspaces cannot supply this evidence because all workloads there use serverless compute; retain the classic report separately with its own workspace, cloud, region, package digest, and cleanup ledger. require_classic_compute: true remains available for a full run in a classic workspace that also carries every other authoring fixture.

Delta Sync and provisioned throughput also have independent, destructive gates. Configure DATABRICKS_AUTHORING_DELTA_SYNC_SPEC and dispatch with require_delta_sync: true, or configure DATABRICKS_AUTHORING_PROVISIONED_THROUGHPUT_SPEC and dispatch with require_provisioned_throughput: true. Each required check creates a uniquely named endpoint, verifies it, creates or updates the governed resource, deletes it, verifies deletion, and runs the post-certification sweeper. Provisioned throughput can incur material cost and must not be enabled without an approved workspace fixture and budget.

Workflow inputs select execution as well as promotion. The protected job exposes Agent Mode, classic-compute, Delta Sync, and provisioned-throughput fixtures to the certification process only when their corresponding input is enabled. A stale environment variable therefore cannot silently run an incompatible or cost-bearing optional lifecycle. The classic-compute scope similarly masks all full-suite resource fixtures and sweep roots.

DATABRICKS_AUTHORING_AUTH_MODE is deliberately separate from the base certification identity's DATABRICKS_AUTH_MODE: protected CI can keep its app-scoped OAuth M2M identity for deployment and crash cleanup while the authoring lifecycle proves execution with the short-lived OBO user token. When run_authoring is selected, the workflow appends all eight authoring checks to the required evidence set; a configured-but-optional lifecycle is not sufficient for capability promotion.

Release evidence workflow

For release evidence, dispatch Databricks authoring certification against the protected databricks-live GitHub Environment. This focused workflow requires the OBO identity and every fixture. Pass a successful same-commit Live integration tests run id, or let the scheduled workflow resolve it. Tier A downloads that run's exact package candidate instead of repacking, runs the candidate tarball's own compiled certifier through every destructive lifecycle, asserts that every required result passed with an empty cleanup ledger, sweeps before and after the run, and retains the Tier R evidence, Tier A evidence, shared tarball, and cross-tier manifest for 90 days. Its concurrency group does not cancel an in-flight run, so a newer dispatch cannot interrupt cleanup. Refresh the short-lived DATABRICKS_OBO_TOKEN immediately before dispatching it. Host/token identity validation runs before install or build; a failure at that preflight cannot create resources and therefore skips the post-run sweeper and artifact upload instead of emitting secondary missing-build failures.

For a native-SDK major, old evidence is intentionally insufficient. The retained evidence must name the exact package candidate and artifact digest, include sdkJs in its API-version map, and be captured after the generated-client migration. Do not promote or publish the candidate from unit tests alone.

Run the sweeper in dry-run mode first:

node scripts/sweep-databricks-authoring-certification.mjs
FABRIC_DATABRICKS_SWEEP_DELETE=1 \
  node scripts/sweep-databricks-authoring-certification.mjs

Workspace-object cleanup fails before discovery when the explicit raw protocol client is unavailable; after validation, the same captured client is used for every listed deletion. Missing resources remain safe no-ops, while any real deletion failure leaves the gate red with the leaked resource ID.

For a packed package, packageArtifactSha256 is the standard SHA-256 digest of the .tgz bytes and must match sha256sum on the retained package. A directory target uses a deterministic tagged tree digest that binds each relative path and file length. Directory hashing rejects symbolic links and other non-file entries, preventing a certification target from escaping its tree or introducing a cycle. The workflow retains the package and report together so this relationship is machine-verifiable after download.

validate-databricks-cross-tier-evidence.mjs fails when Tier R and Tier A differ by package version, source commit, tarball digest, required-check result, or freshness. This means a weekly authoring success cannot be combined with a separately packed consumption candidate to support a first-class claim.

Latest authoring evidence boundary

The current npm package is 7.0.2, and it has byte-identical same-package Tier R and Tier A evidence from commit 544cd478d29b26ed7090499c682af0922f4533d1. The focused authoring run 31056050505 downloaded the exact tarball retained by Tier R run 31053240003. Evidence dbx-cert-37f20b775f8f1e676f6b7d36bb17bb61f1ae9c593e560e028d1872764d70693d records all ten required checks as passed in Azure eastus2 under Databricks U2M/OBO:

• approval provenance;
• serverless Jobs, Lakeflow, direct-vector AI Search, and custom-model Serving lifecycles;
• managed-only Unity Catalog administration with OBO execution;
• Workspace object and secret-reference lifecycles; and
• Genie Agent create, query, update, ACL-read, and trash.

The retained package digest is edf67563b0bf0daa211ada78c5a3996063a39befa464fa2df27ba04e102cb59c. The workflow independently matched it to the packed tarball, verified every required result, required cleanupRequired: false after each destructive lifecycle, and completed empty pre/post sweeps. Cross-tier manifest databricks-cross-tier-certification.json binds it byte-for-byte to Tier R run 31053240003 and the same commit. This closes the same-candidate authoring proof for 7.0.2. It does not claim classic compute in westus3, Delta Sync, provisioned throughput, or Genie Agent Mode authoring.

Prior 4.10.0 authoring evidence

Published package 4.10.0 retains an earlier complete same-package authoring record from run 30058616867. Evidence dbx-cert-0574345bfe1bb7fe7b5cf59d862fcabd0c74ea290fde86e95ac453ac7251b79b passed the same ten required checks with package digest a688c1bd52ccbd7aad923d8c4212c6b305b080663877be289d5e51838d6a20cf and an empty cleanup ledger. It remains historical evidence for that exact package.

Prior 4.0.2 authoring evidence

The exact @fabric-harness/databricks@4.0.2 package passed two protected Azure gates from commit 0961a17b1349213c4c0193ef446639955d624de5:

• GitHub Actions run 29660486290 retained evidence dbx-cert-b2c709717ef3a452234a5b2944311a576dafda03a9eea8246747ab3fa430b183. All 11 required identity, approval, serverless Jobs, Lakeflow, direct-vector AI Search, Delta Sync, custom-model Serving, UC OBO, Workspace, secrets, and Genie checks passed in Azure eastus2. Genie verification included a governed query and ACL decoding through the official Access Management SDK.
• GitHub Actions run 29660486237 retained evidence dbx-cert-d5bdbd276533a8634910e223898337314446561be2a444030fc2d11baa237bec. Identity, approval provenance, and the policy-bound classic new_cluster Jobs definition lifecycle passed in Azure westus3 without starting the cluster.

Both reports record SDK JS 0.21.0 and package digest 72f602626b9d79a808f6dd9e30f6ba703061bda19bae2aef983e7e92d7c61a45, independently matched to sha256sum on the retained tarball. Every destructive lifecycle completed all six phases with cleanupRequired: false, and every pre/post sweep was empty. This evidence does not promote Provisioned Throughput or Agent Mode Beta: the former has no configured fixture, and the protected workspace still returns FEATURE_DISABLED for the latter.

Prior 4.0.1 promotion evidence

The exact @fabric-harness/databricks@4.0.1 package passed two protected Azure gates from commit dce71eaeac1edba47b4122cd5d81ccf0a1110f64:

• GitHub Actions run 29657106492 retained evidence dbx-cert-7b2a371109ee02a47b4a5cf9403eb6aafbf2420ea6d3194de574cfc6a941a7f7. All 11 required identity, approval, serverless Jobs, Lakeflow, direct-vector AI Search, Delta Sync, custom-model Serving, UC OBO, Workspace, secrets, and Genie checks passed in Azure eastus2.
• GitHub Actions run 29657107378 retained evidence dbx-cert-69817b237074b4fad5e86ee2aec7733899b0d08e525744ca0bf5cad173421017. Identity, approval provenance, and the policy-bound classic new_cluster Jobs definition lifecycle passed in Azure westus3 without starting the cluster.

Both reports record SDK JS 0.21.0 and package digest e805d79ae2b5f50e918eacb7eac2c7ca16b24da98e9a8d9d69e0653c43abd2a0, independently matched to sha256sum on the retained tarball. Every destructive lifecycle completed all six phases with cleanupRequired: false; Genie executed a governed query; and every pre/post sweep was empty.

Prior 4.0.0 promotion evidence

The exact @fabric-harness/databricks@4.0.0 package passed two protected Azure gates from commit 7614a3d81a69cacaf84b13c716457b5655d67abd:

• GitHub Actions run 29640438183 retained evidence dbx-cert-d4d3095389c096994a89d56560b62af1b5e396c7831599ea1fd67d1ad1644f79. All 11 required identity, approval, serverless Jobs, Lakeflow, direct-vector AI Search, Delta Sync, custom-model Serving, UC OBO, Workspace, secrets, and Genie checks passed in Azure eastus2.
• GitHub Actions run 29640438874 retained evidence dbx-cert-a85de02baf1b50717dbb070356006c2185a3bd7353b91eccda089a89a5984a82. Identity, approval provenance, and the policy-bound classic new_cluster Jobs definition lifecycle passed in Azure westus3 without starting the cluster.

Both reports record SDK JS 0.21.0 and package digest 531f86f4c8d1cecf3df9f9d7544a6341a9379562abe9e135c742ee6f51fa969b, independently matched to sha256sum on the retained tarball. Every destructive lifecycle completed all six phases with cleanupRequired: false; Genie executed a governed query; and every pre/post sweep was empty.

Prior 3.0.3 promotion evidence

The exact @fabric-harness/databricks@3.0.3 package passed the protected Azure eastus2 gate in GitHub Actions run 29633534464. Evidence dbx-cert-793b88b91f44e5deb0eeff2e2ba168666bb2d03fd2883da256f25c269d3cb73c records SDK JS 0.21.0 and package digest 9b87c9447d865b84a55a77232ed61eeb8d7f175155ab4db11fc874c59c287b84.

Prior 3.0.2 promotion evidence

The exact @fabric-harness/databricks@3.0.2 package passed the protected Azure eastus2 gate in GitHub Actions run 29632616112. Evidence dbx-cert-a7c0f49484d6f3d2409be4f6a6b196fc4a1ae83cdcf00ce89de0c8cf4fcab19c records SDK JS 0.21.0 and package digest f0330b2ef874995d60b1107b0dc7d02a0dbca24326d5ed867e357289b4e8b910; the workflow independently matched it to sha256sum on the retained tarball.

All 10 required checks passed. Every destructive lifecycle completed all six phases with cleanupRequired: false, Genie executed a governed query, and the post-run sweep found no retained resources. The pre-run sweep also proved crash recovery: it discovered and deleted the serving endpoint left when the preceding run's short-lived OBO token expired. The failed run therefore did not require manual workspace cleanup.

Prior 3.0.1 promotion evidence

The exact @fabric-harness/databricks@3.0.1 package passed the protected Azure eastus2 gate in GitHub Actions run 29631548469. Evidence dbx-cert-8342f00839b3ed0bbe7ef93483207d0a5e4ea36c5a80d6c9854c1681bccfb930 records SDK JS 0.21.0 and package digest e16c6524586742f985a840f5e5a10b82f0ac6125e8f61a75e212d22eace47260. The workflow independently matched that value to sha256sum on the retained .tgz.

All 10 required identity, approval, and lifecycle checks passed. Jobs, Lakeflow, AI Search, custom-model Serving, managed Unity Catalog schema/grants with OBO execution, Workspace objects, secret references, and Genie completed create, verify, mutate, verify-mutation, delete, and verify-delete. Genie also executed a governed query. Every lifecycle cleanup ledger was empty, and both independent sweeps discovered zero retained resources.

The evidence promotes only the recorded Azure region and tested variants. It does not promote classic compute, delta-sync, provisioned throughput, Agent Mode streaming, or another cloud/region.

Historical promotion evidence

The records below certify earlier implementations. They remain useful behavioral evidence, but the current native-SDK promotion claim comes only from the exact 4.4.1 package evidence above.

The eight protected management capability families are stable for their documented scope after the exact 2.0.0 package candidate from commit c3d0c9091db39cca6bee5736922d70891aefcfdc passed all ten required checks with no cleanup leaks in GitHub Actions run 29622641051. Evidence dbx-cert-db47ec980bd8e7bdd9687896c68530d121f5dcac0421cf7c1f85d07a49ff52ef records the package artifact digest 35f492e0ff28cee9a4eee9e719a2d90fe9bd312209e7cd2d128f046dfbe8e087; both the pre-run and post-run destructive sweeps found no retained certification resources. The retained workflow artifact expires after 90 days.

The broader protected release run 29622641870 then passed all 33 required checks from the same commit and package version. Its retained evidence dbx-cert-0f987b096a5015fb17387564c10ffe1b4356e3256610e17350af57d4593ec1ed records those 33 required passes plus the separately configured optional OBO check (34 total), zero failures, and successful cleanup for every destructive lifecycle. The two unconfigured optional checks were Agent Services and Genie Agent Mode; neither is included in this production claim. The run also passed Databricks App recovery, load, and black-box lifecycle conformance. The temporary release-branch allowance was removed from the protected environment after evidence upload.

That evidence used serverless Jobs, a direct-vector index, and custom-model serving on the protected Azure eastus2 workspace with OBO execution; it does not imply classic compute, delta-sync, provisioned throughput, Agent Mode, or another cloud/region passed the same live gate. Provisioned Throughput, AI Gateway administration, Agent Mode streaming, and Databricks App OBO authentication retain their separate Databricks preview constraints. Contract tests alone never promote a capability or claim another cloud/region.

Capability metadata separates contractClouds (typed API targets) from clouds (clouds represented by linked retained live evidence). Do not use contract portability as proof that an unlisted cloud/region passed certification.