Databricks connectors and sandboxes

Databricks appears in both the sandbox and connector layers, but those layers solve different problems. Select them by capability rather than by provider name.

flowchart TD
  NEED{What must the agent do?}
  NEED -->|Run governed SQL| SQL[Databricks SQL sandbox or SQL tool]
  NEED -->|Read or write governed files| VOL[Unity Catalog Volume connector]
  NEED -->|Read workspace source files| WS[Workspace Files source]
  NEED -->|Persist user attachments| ATT[UC Volumes attachment store]
  NEED -->|Run bash, packages, or arbitrary code| GEN[Docker or remote code sandbox]

  SQL --> WH[SQL Warehouse]
  VOL --> FILES[Files API]
  WS --> WAPI[Workspace API]
  ATT --> FILES
  GEN --> DATA[Call Databricks tools over governed APIs]

  classDef question fill:#fef3c7,stroke:#d97706,color:#422006
  classDef fabric fill:#dbeafe,stroke:#2563eb,color:#172554
  classDef dbx fill:#dcfce7,stroke:#16a34a,color:#052e16
  class NEED question
  class SQL,VOL,WS,ATT,GEN fabric
  class WH,FILES,WAPI,DATA dbx

SQL sandbox

databricksSqlSandbox() adapts session.shell(sql) to SQL Statement Execution on a SQL Warehouse. It returns JSONL or CSV in stdout. Its small filesystem is in-memory session storage; it is not DBFS, a Volume, a cluster driver, or an operating-system shell.

import { databricksSqlSandbox } from '@fabric-harness/databricks/sql-sandbox';
import { init } from '@fabric-harness/sdk';

const runtime = await init({
  sandbox: databricksSqlSandbox({
    host: process.env.DATABRICKS_HOST!,
    principal: { kind: 'on-behalf-of', userToken: () => workspaceIdentity() },
    warehouseId: process.env.DATABRICKS_WAREHOUSE_ID!,
    catalog: 'main',
    schema: 'analytics',
    resultFormat: 'jsonl',
  }),
});

const session = await runtime.session();
const result = await session.shell(`
  SELECT region, sum(net_revenue) AS revenue
  FROM orders
  GROUP BY region
  ORDER BY revenue DESC
`);

The SDK backend name is also usable after registering the Databricks provider adapter:

import { registerDatabricksSqlSandboxBackend } from '@fabric-harness/databricks';
import { init } from '@fabric-harness/sdk';

registerDatabricksSqlSandboxBackend();
const runtime = await init({ sandbox: 'databricks' });

The factory reads host, Warehouse, and identity from the sandbox creation environment. A portable databricks-sql ref stores only host, Warehouse, catalog, schema, result format, and timeout; PAT, OAuth secret, OBO token, generated SDK client, and token provider are deliberately excluded. Attach resolves credentials again in the receiving process, so a rotated credential can be used without rewriting the ref. Missing host, Warehouse, or attach-time identity fails closed.

Use databricksSqlReadTool() for model-facing analytics reads. It advertises a read effect and fails locally unless input is one SELECT (a SELECT-ending CTE is accepted); mutations, administration keywords, malformed SQL, and multiple statements never reach Statement Execution.

import {
  analyticsCopilotGovernance,
  databricksSdk,
  databricksSqlReadTool,
} from '@fabric-harness/databricks';

const sdk = databricksSdk({ host, principal });
const sqlRead = databricksSqlReadTool(sdk.statements, {
  warehouseId: process.env.DATABRICKS_WAREHOUSE_ID!,
});
const governance = analyticsCopilotGovernance({
  stewardAudience: 'analytics-stewards',
  catalogs: ['main'],
});

The analytics-copilot pack scopes approval routing to the SQL and Genie services. sql_read and ordinary databricks_genie_ask calls are reads and stay interactive; policy-bound databricksSqlTool() execution and Genie lifecycle writes still route to the steward audience. Enabling a different authoring service fails bundle initialization until approval routing covers it. This Harness check is defense in depth: Unity Catalog privileges under the acting service principal or OBO user remain authoritative.

Use databricksSqlTool(client, executionPolicy, options) only when the model deliberately needs more than SELECT-only access. Bind exact statements or supply a server-side validator; the allowAnyStatement: true arm is an explicit advanced opt-out. Approval routing remains a separate defense-in-depth layer. The sandbox is useful when the session's shell abstraction should itself mean SQL. See the runnable with-analytics-copilot example for service-principal and OBO authentication, expected output, and failure behavior.

Unity Catalog Volume connector

The connector uses the Databricks Files API and keeps all paths under a configured /Volumes/<catalog>/<schema>/<volume> root.

import { databricksVolumeSource } from '@fabric-harness/connectors/databricks-volume';

const source = databricksVolumeSource({
  host: process.env.DATABRICKS_HOST!,
  token: async () => workspaceIdentity(),
  volumePath: '/Volumes/main/support/knowledge',
  include: (path) => path.endsWith('.md') || path.endsWith('.pdf'),
});

await session.mount('/knowledge', source);

databricksVolumeWriter() provides scoped put() and delete() operations. Use UcVolumesAttachmentStore when attachments must participate in Fabric's attachment lifecycle. The acting principal still needs the relevant USE CATALOG, USE SCHEMA, and Volume privileges.

General-purpose code execution

For Python, package installation, bash, repository mutation, or untrusted code, use a code sandbox such as Docker, Kubernetes, E2B, Daytona, Modal, or another remote backend. Give that sandbox Databricks tools or scoped OAuth access as needed. This preserves the common sandbox interface without pretending a SQL Warehouse is a machine shell.

See the focused Databricks SQL sandbox reference and the sandbox matrix.