Databricks integrations

Complete map of Fabric Harness integrations for Databricks data, AI, orchestration, state, governance, telemetry, and cost.

@fabric-harness/databricks exposes composable factories and a databricks() bundle. Use the bundle for the common governed stack; use individual exports when you need a narrower integration.

To scaffold project wiring (managed files, env stubs, dependencies), use Databricks recipes:

fh add databricks sql
fh add lakebase
fh add vector-search
fh add lakeflow

Data and AI

Service	Fabric Harness API	Agent use
Unity AI Gateway	`databricksFoundationModelProvider()`, `ensureDatabricksAiGateway()`, `listDatabricksModelServices()`	Discover and invoke `system.ai.*` model services with submission/tenant request tags
Model Serving	`databricksFoundationModelProvider({ mode: 'serving-endpoints' })`	Custom serving-endpoint inference
SQL Warehouse	`databricksSqlTool()`	Parameterized, policy-visible SQL statements
Unity Catalog	`unityCatalogTablesTool()`, `databricksTableInfoTool()`	Discover governed tables and metadata
Vector Search	`databricksVectorSearch()`, `databricksRagChain()`	Retrieval for managed- or self-managed-embedding indexes; cookbook online chain
RAG quality	`scoreRagTurn()`, MLflow 3 export, managed evaluation Job	Local smoke checks plus a UC evaluation dataset and Databricks managed relevance, groundedness, sufficiency, and correctness judges
Embeddings	`databricksEmbeddings()`	Query embeddings through a serving endpoint
AI Functions	`databricksAiQueryTool()`	Invoke `ai_query()` through a SQL Warehouse
AI/BI Genie	`databricksGenieTool()`	Natural-language analytics over a configured Genie space
Feature Serving	`databricksFeatureLookupTool()`	Low-latency governed feature lookup
Workspace files	`databricksWorkspaceSource()`	Read-only agent context from workspace files

Inference names and URLs

Use the workspace origin for DATABRICKS_HOST, not an API path. Fabric selects the inference base from the model name:

Model value	Mode	OpenAI-compatible base URL
`system.ai.gpt-oss-20b` (or another discovered `system.ai.*` service)	Unity AI Gateway	`${DATABRICKS_HOST}/ai-gateway/mlflow/v1`
A custom endpoint name such as `support-agent-prod`	Custom Model Serving	`${DATABRICKS_HOST}/serving-endpoints`

DATABRICKS_INFERENCE_MODE=auto is the default. Set ai-gateway or serving-endpoints only when overriding automatic routing. Use DATABRICKS_AI_GATEWAY_BASE_URL for a proxy or explicitly configured Gateway base. Discover services enabled in the current workspace with listDatabricksModelServices(client) instead of assuming every system.ai.* service is available.

Engineering and operations

Service	Fabric Harness API	Agent use
Jobs	`databricksRunJobTool()`	Start an existing Databricks Job with idempotent retries
Notebooks	`databricksNotebookTool()`	Submit a one-time notebook run
Lakeflow	`databricksLakeflowTools()`	List, inspect, start, and stop pipelines
MLflow runs	metric and parameter tools	Write run metadata
MLflow tracing	`mlflowTraceExporter()`	Export settled agent spans to MLflow Tracing
Serving usage	`servingUsageCapture()`	Associate inference-table usage with submissions
System tables	`databricksConsumption()`	Aggregate billable usage for tenants and agents
Actual-cost budgets	`databricksActualCostSource()`, `databricksTenantCostLimit()`	Reconcile policy budgets against usage records
Lakebase	`lakebaseClient()`, `databricksPersistence()`	Sessions, submissions, conversation streams, and telemetry
UC Volumes	`UcVolumesAttachmentStore`, Volume source/writer	Governed attachments and non-tabular context

Add a standalone tool

import {
  DatabricksRestClient,
  databricksFeatureLookupTool,
  databricksIdentity,
  withGovernance,
} from '@fabric-harness/databricks';

const identity = databricksIdentity({
  kind: 'service-principal',
  host: process.env.DATABRICKS_HOST!,
  clientId: process.env.DATABRICKS_CLIENT_ID!,
  clientSecret: process.env.DATABRICKS_CLIENT_SECRET!,
});

const client = new DatabricksRestClient({
  host: process.env.DATABRICKS_HOST!,
  token: identity,
});

const customerFeatures = withGovernance(
  databricksFeatureLookupTool(client, {
    endpoint: 'customer-features',
    name: 'lookup_customer_features',
  }),
  {
    principal: `sp:${process.env.DATABRICKS_CLIENT_ID}`,
    onLineage: (record) => auditSink.write(record),
  },
);

Integration boundaries

Fabric Harness does not replace Unity Catalog permissions or Databricks resource ACLs.
The SQL tool and SQL sandbox execute statements; they do not provide a Linux shell.
Workspace files and Unity Catalog Volumes are separate APIs and path spaces.
databricks-serving is a proxy target, while databricks-app hosts the Node runtime.
Availability and API behavior can vary by cloud, region, workspace feature enablement, and preview status. Validate each enabled integration in the target workspace.