Enterprise Databricks adoption guide

Fabric Harness helps a Databricks customer turn governed data and AI services into durable agent applications. It is most useful when an application spans several Databricks services, a model may choose actions, identity must follow the user or service, or operations need approval and audit evidence.

Databricks remains the system of authority for workspace identity, Unity Catalog permissions, compute, data, model endpoints, retrieval, orchestration, and storage. Fabric owns the application control loop around those services: sessions, tools, policy, approvals, recovery, interoperability, deployment artifacts, lineage, and cost correlation.

Choose one lighthouse workload

Start with one workload whose value and risk can both be measured. Avoid beginning with a generic assistant that can call every workspace capability.

The workload map lists the supported API, managed-recipe, and native escape-hatch paths. The why Fabric page explains when the official Databricks SDK alone is the better choice.

Reference architecture

flowchart LR
  USER[Employee, application, or event] --> EDGE[Databricks App or authenticated API]
  EDGE --> ID[App principal, OAuth M2M, or OBO user]
  ID --> RUNTIME[Fabric agent runtime]

  subgraph Controls[Fabric application controls]
    RUNTIME --> POLICY[Tools, policy, approvals]
    RUNTIME --> STATE[Durable session and submission state]
    RUNTIME --> OBS[Lineage, traces, cost correlation]
  end

  POLICY --> GATEWAY[Unity AI Gateway]
  POLICY --> SQL[SQL Warehouse and Unity Catalog]
  POLICY --> SEARCH[AI Search and Genie]
  POLICY --> WORK[Jobs and Lakeflow]
  STATE --> LAKEBASE[Lakebase]
  STATE --> VOLUMES[UC Volumes]
  OBS --> MLFLOW[MLflow and system tables]

  classDef fabric fill:#dbeafe,stroke:#2563eb,color:#172554
  classDef control fill:#fef3c7,stroke:#d97706,color:#422006
  classDef dbx fill:#dcfce7,stroke:#16a34a,color:#052e16
  class RUNTIME fabric
  class POLICY,STATE,OBS control
  class GATEWAY,SQL,SEARCH,WORK,LAKEBASE,VOLUMES,MLFLOW dbx

This architecture does not require exposing a generic REST client to the model. Stable platform operations use exact-pinned official Databricks SDK clients. Model-facing tools are a smaller, policy-visible surface with read, write, execute, or external effects.

Select the acting identity

Identity is an architecture decision, not a deployment detail.

Do not accept a user id from a request body and call it OBO. The runtime must derive the principal from Databricks-authenticated ingress or a verified token. Tokens remain in credential providers and request headers; prompts, tool arguments, lineage, and cost labels contain non-secret identity labels only. See local naming and authentication and enterprise controls.

Establish the workspace landing zone

Before application development, the platform team should provide:

1. A development workspace and disposable certification namespace.
2. A dedicated OAuth service principal, App principal, or OBO test users for each selected mode.
3. A SQL Warehouse and explicit UC catalog, schema, table, volume, and function grants.
4. An enabled system.ai.* AI Gateway service or approved custom Model Serving endpoint.
5. AI Search, Genie, Feature Serving, Jobs, or Lakeflow fixtures only for workloads that use them.
6. A Lakebase endpoint, database role, schema owner, migrations, and App DML grants when state must survive restart.
7. UC Volumes for governed attachments when the application accepts files.
8. System-table access and a cost attribution key when actual-cost reconciliation is required.
9. Private connectivity, DNS, CA, and outbound policy appropriate to the workspace network design.
10. Separate steward audiences for approvals and operators for deployment/runtime administration.

Feature availability varies by cloud, region, account, and workspace enablement. Read the compatibility matrix before treating an API contract as live evidence for a specific customer environment.

Roll out in evidence-producing phases

Phase 0 — contract and threat model

• Name the business owner, data owner, platform owner, approval audience, and incident owner.
• List the exact tools the model can select and classify every effect.
• Define data boundaries, retention, deletion, RTO/RPO, cost limits, and required evaluation scores.
• Decide what remains deterministic application code through bundle.sdk and what becomes a model tool through bundle.tools.

Exit when architecture and security reviewers can identify the acting principal, every reachable Databricks resource, every mutation, and the recovery path.

Phase 1 — local and contract proof

• Start with fh init --template databricks or add the narrow managed recipes.
• Run deterministic mocks for tool assembly, policy, input/output validation, and HTTP routes.
• Prove unsafe SQL, unapproved writes, missing credentials, and unsupported configuration fail before a platform mutation.
• Package the exact candidate versions and inspect the build manifest and SBOM.

Local tests prove application behavior. They do not prove workspace permissions, preview availability, network reachability, or restart durability.

Phase 2 — protected workspace proof

• Run the Tier R consumption certifier with the same package version intended for deployment.
• Exercise allowed and denied UC paths, AI Gateway, the selected data/AI services, volumes, Lakebase, lineage, system tables, actual-cost reconciliation, and the App.
• Run Tier A separately in a disposable namespace when agents can author resources.
• Retain secret-redacted evidence, exact package version, commit, artifact digest, cloud, region, auth mode, and cleanup results.

Required checks must pass. A skipped Tier R capability is a failed release proof, not a green run. Tier O previews may remain unconfigured or fail without blocking Tier R unless the deployment explicitly promotes them.

Phase 3 — pilot

• Limit users, catalogs, tools, concurrency, and daily spend.
• Compare answers and actions with a golden set and human review.
• Measure latency, completion, refusal, approval, retry, and cost behavior.
• Restart and redeploy the App; verify session, submission, stream, and attachment recovery.
• Test tenant deletion, token expiry, endpoint unavailability, UC denial, and delayed system-table accounting.

Before admitting a user to a shared App, combine the retained target-workspace artifacts into one fail-closed pilot report. This gate requires the exact package and App bytes, all Tier R checks, restart recovery, hook-authored dynamic-agent recovery, two distinct user identities, and one complete OBO expiry/refresh lifecycle:

pnpm databricks:pilot:verify -- \
  --evidence artifacts/databricks-certification.json \
  --package artifacts/release-packages/fabric-harness-databricks.tgz \
  --app artifacts/certified-databricks-app/databricks-app \
  --recovery artifacts/databricks-app-recovery.json \
  --dynamic artifacts/databricks-dynamic-agent.json \
  --isolation artifacts/databricks-app-user-isolation.json \
  --obo-lifecycle artifacts/databricks-obo-lifecycle.json \
  --pilot-id finance-support-pilot \
  --enabled-checks databricks-app,dynamic-agent,sql,genie \
  --expected-version "$DATABRICKS_PACKAGE_VERSION" \
  --expected-commit "$DATABRICKS_CERTIFIED_COMMIT" \
  --expected-workspace-id "$DATABRICKS_WORKSPACE_ID" \
  --expected-cloud azure \
  --expected-region eastus2 \
  --output artifacts/databricks-controlled-pilot.json

Name the pilot for its function, list only the Tier R checks it will enable, and replace the workspace, cloud, and region values with the target workspace values. A preview or optional check cannot enter this report merely because it exists; it needs its own retained evidence. A passing command prints Databricks controlled-pilot evidence passed and writes a secret-free report with status controlled-pilot-ready. It deliberately sets productionCertified: false: the report authorizes only the named pilot and does not establish the separate 14-day claim. Missing, stale, digest-mismatched, cross-region, single-user, or incomplete OBO-refresh evidence fails without writing a readiness report.

Generate databricks-obo-lifecycle.json with the prepare/verify sequence in Local naming and authentication. Download the other artifacts from the protected target-workspace certification run. The verifier reads them locally and performs no cloud mutations. After the pilot proof, remove the downloaded bearer-free artifacts according to the customer's evidence-retention policy; resource cleanup remains the responsibility of the certification and deployment runbooks.

Phase 4 — production

• Promote an immutable artifact already linked to passing environment evidence.
• Apply Lakebase migrations with a stable schema owner before App rollout.
• Monitor App health, model/SQL usage, approval queues, failed observers, and cost drift.
• Re-certify on package, SDK, cloud/region, auth, App resource, or material workspace-policy changes.
• Keep preview capabilities separately labeled and independently promotable.

Operating ownership

Fabric supplies correlated evidence, but it does not transfer ownership away from these teams.

Production decision checklist

• The workload uses a documented first-party path or an explicitly owned native escape hatch.
• The acting principal and tenant derive from verified authentication.
• UC and resource grants are least privilege and include a tested denial case.
• The model receives only bounded tools; deterministic admin code stays out of the tool set.
• Every write/execute path has explicit policy and, where required, exact-input approval.
• Secrets never enter prompts, tool inputs, logs, evidence, or persisted user messages.
• Lakebase migrations, App grants, backup, restore, restart, and deletion are proven.
• AI Search/Genie answers, SQL, citations, and MLflow evaluation outputs are inspectable.
• Estimated budgets act immediately and actual costs reconcile from system tables.
• Package, commit, artifact digest, cloud, region, auth mode, and evidence all match.
• Preview and workspace-dependent capabilities are labeled and have an owner.
• Failure, rollback, and incident runbooks have been exercised.

Build the first proof

For an analytics copilot, begin with the credential-free with-analytics-copilot example, then connect one Warehouse and one Genie Agent. For a stateful application, start with the Databricks quickstart, add Lakebase, and follow the Databricks App deployment guide. For RAG, use the Databricks RAG guide and treat MLflow evaluation plus restart recovery as part of the release proof.

Continue with development, enterprise controls, and live certification.