Databricks architecture

How Fabric Harness routes identity, policy, model calls, data tools, durable state, telemetry, and deployments through Databricks.

Fabric Harness separates the agent runtime from the services the agent is allowed to call. A single Databricks principal can be threaded through Model Serving, REST tools, SQL, and Lakebase credential exchange. Fabric policy adds approvals, egress restrictions, budgets, and audit correlation; Unity Catalog still makes the final data authorization decision.

Runtime layers

Text alternative and Mermaid source

Diagram flow: fh run; Jobs and agents API; Schedule or channel; Job or persistent agent definition; Session and model loop; Capability policy; Human approval; Audit, lineage, and cost; App service principal; On-behalf-of user; OAuth M2M; Model Serving.

flowchart TB
  subgraph Entry[Invocation]
    CLI[fh run]
    API[Jobs and agents API]
    EVT[Schedule or channel]
  end

  subgraph Runtime[Fabric Harness runtime]
    DEF[Job or persistent agent definition]
    SES[Session and model loop]
    POL[Capability policy]
    APR[Human approval]
    AUD[Audit, lineage, and cost]
  end

  subgraph Identity[Databricks identity]
    APP[App service principal]
    OBO[On-behalf-of user]
    M2M[OAuth M2M]
  end

  subgraph Services[Databricks services]
    SERVE[Model Serving]
    SQL[SQL Warehouse]
    UC[Unity Catalog]
    RAG[AI Search]
    STATE[Lakebase]
    OPS[MLflow and system tables]
  end

  CLI --> DEF
  API --> DEF
  EVT --> DEF
  DEF --> SES
  SES --> POL
  POL -->|sensitive action| APR
  POL -->|allowed action| Services
  SES --> AUD
  APP --> SES
  OBO --> SES
  M2M --> SES
  SERVE --> UC
  SQL --> UC
  RAG --> UC
  STATE --> AUD
  OPS --> AUD

  classDef entry fill:#f4f4f5,stroke:#71717a,color:#18181b
  classDef fabric fill:#dbeafe,stroke:#2563eb,color:#172554
  classDef identity fill:#fef3c7,stroke:#d97706,color:#422006
  classDef dbx fill:#dcfce7,stroke:#16a34a,color:#052e16
  class CLI,API,EVT entry
  class DEF,SES,POL,APR,AUD fabric
  class APP,OBO,M2M identity
  class SERVE,SQL,UC,RAG,STATE,OPS dbx

One request, one governed identity

Text alternative and Mermaid source

Diagram flow: autonumber; actor User; participant App as Databricks App; participant Fabric as Fabric Harness; participant Policy as Policy and approvals; participant Model as Model Serving; participant Tool as SQL or Databricks API; participant UC as Unity Catalog; participant State as Lakebase; User leads to >App: Prompt or job input; App leads to >Fabric: Request plus app or user identity; Fabric leads to >State: Load session and submission.

sequenceDiagram
  autonumber
  actor User
  participant App as Databricks App
  participant Fabric as Fabric Harness
  participant Policy as Policy and approvals
  participant Model as Model Serving
  participant Tool as SQL or Databricks API
  participant UC as Unity Catalog
  participant State as Lakebase

  User->>App: Prompt or job input
  App->>Fabric: Request plus app or user identity
  Fabric->>State: Load session and submission
  Fabric->>Model: Prompt with short-lived OAuth token
  Model-->>Fabric: Tool request
  Fabric->>Policy: Evaluate capability and approval rules
  alt approval required
    Policy-->>User: Approval request
    User-->>Policy: Approve or deny
  end
  Policy->>Tool: Execute as the governed principal
  Tool->>UC: Enforce catalog, schema, table, row, and column grants
  UC-->>Tool: Authorized result or denial
  Tool-->>Fabric: Redacted result plus lineage metadata
  Fabric->>State: Persist events and final result
  Fabric-->>User: Typed response or durable receipt

Finite jobs and persistent agents

Finite jobs live in .fabricharness/jobs/, return one typed result, and are invoked at POST /jobs/:name. Persistent agents live in .fabricharness/agents/, retain an addressable conversation, and return a submission receipt from POST /agents/:name/:id. Both use the same model, tool, policy, and Databricks integration surfaces.

Lakebase can back sessions, submissions, and conversation streams. The runtime exchanges a workspace OAuth token for a short-lived database credential, supplies that credential through the Postgres pool, refreshes early, and deduplicates concurrent refreshes.

Deployment boundaries

databricks-app runs the bundled Node server inside Databricks Apps. databricks-serving builds an MLflow ResponsesAgent proxy that calls an agent hosted elsewhere; it does not execute the TypeScript runtime inside Model Serving. The App exposes the same /responses contract directly. The proxy preserves response.output_text.delta and response.output_item.done events and returns a normal ResponsesAgentResponse for non-streaming clients. See Databricks deployment for target details.