Enterprise Databricks controls

Fabric Harness adds runtime controls around Databricks calls. Those controls are defense in depth: Unity Catalog and Databricks resource permissions remain authoritative.

Identity modes

Token providers are resolved at call time and refresh before expiry. Identity labels, not tokens, are placed in actors, submissions, lineage, and cost records.

For Databricks Apps ingress, use databricksAppsOidcAuthenticator() as the Node server authenticate hook. It validates workspace JWT signatures, issuer, audience, expiry, and signing-key rotation before mapping the forwarded user email to both the Fabric actor and Unity Catalog principal. See Authentication and RBAC.

When Databricks Apps forwards an OBO access token instead of an OIDC identity token, use the narrow Apps authorization entrypoint. It validates the token against the workspace current-user API, returns only safe identity and lifetime metadata, assigns an opaque tenant per user, and caches by a token digest bounded by token expiry:

import { createDatabricksAppUserAuthenticator } from
  '@fabric-harness/databricks/app-user-authorization';
import { startDevServer } from '@fabric-harness/node';

const authenticate = createDatabricksAppUserAuthenticator({
  host: process.env.DATABRICKS_HOST!,
});

await startDevServer({ authenticate });

Missing forwarded authorization falls through so another configured authenticator may run. Invalid or unauthorized forwarded tokens fail closed. The raw token is never returned, logged, or used as a cache key. Applications that need authenticated user identity but do not pass an OBO token to downstream Databricks APIs can opt into trustForwardedUserIdentity. It maps the integrity-protected x-forwarded-user, x-forwarded-email, and x-forwarded-preferred-username ingress headers to a natural-person principal. Generated Databricks App bundles enable this trust mode because they are reachable only through Apps ingress; a forwarded email or preferred username is required so an M2M caller with only x-forwarded-user is not reclassified as a person. trustForwardedUserIdentity, trustForwardedServicePrincipal, appPrincipalId, and appAuthorizedCallerIds are explicit ingress-trust controls; enable them only when the server is exclusively reachable through Databricks Apps. See the runnable examples/with-databricks-simple/app-auth.ts path for deterministic output and failure behavior.

Enforcement flow

flowchart LR
  R[Tool request] --> A[Definition and invocation policy]
  A --> E{Allowed?}
  E -->|No| DENY[Reject and audit]
  E -->|Yes| H{Approval required?}
  H -->|Yes| WAIT[Durable approval wait]
  WAIT --> DEC{Decision}
  DEC -->|Deny or expire| DENY
  DEC -->|Approve| CALL[Call Databricks]
  H -->|No| CALL
  CALL --> UC[Unity Catalog and resource ACLs]
  UC -->|Denied| DENY
  UC -->|Allowed| RESULT[Redacted result]
  RESULT --> LINEAGE[Lineage, telemetry, and cost]

  classDef decision fill:#fef3c7,stroke:#d97706,color:#422006
  classDef allow fill:#dcfce7,stroke:#16a34a,color:#052e16
  classDef deny fill:#fee2e2,stroke:#dc2626,color:#450a0a
  class E,H,DEC decision
  class CALL,UC,RESULT,LINEAGE allow
  class DENY deny

Controls to configure

1. Use a least-privilege service principal or a user OBO token for each request.
2. Restrict outbound network policy to the workspace host and any explicitly required services.
3. Add catalog allowlists as defense in depth; do not treat them as a substitute for UC grants.
4. Route write and execute tools to a data-steward approval audience.
5. Persist sessions, submissions, and streams in Lakebase for restart recovery.
6. Send governed tool lineage to an audit sink, MLflow, OpenTelemetry, or Lakebase telemetry tables.
7. Apply estimated and actual-cost tenant budgets, then reconcile against system-table usage.
8. Keep secrets in Databricks App resources, environment-backed secret references, or an external secret manager. Never put credentials in prompts, tool inputs, or lineage labels.
9. Build with provenance and SBOM output, inspect the v2 manifest, and require an API token on production artifact routes.

Lakebase credential exchange

lakebaseClient() does not use a workspace OAuth token as the Postgres password. It exchanges the workspace token for a database credential using the endpoint resource name, refreshes that credential early with jitter, single-flights concurrent refreshes, and supplies it to the connection pool.

const app = databricksApp();
const server = await startDevServer({
  host: '0.0.0.0',
  port: Number(process.env.DATABRICKS_APP_PORT ?? 8080),
  ...(await app.serverOptions()),
});

serverOptions() injects the Lakebase session, submission, and conversation-stream stores into the shared Node server. See Lakebase for configuration.

Schema ownership in automated deployments

A production deployment usually has two database principals:

• the release principal used by CI to deploy resources and apply schema changes;
• the App principal used by the running Databricks App to read and write durable state.

Keep one release principal as the stable owner of the Harness tables. Apply new migrations with that principal before rolling out an App version, then grant the App principal only the DML and sequence permissions it needs. Do not transfer table ownership to every new deployment identity.

-- Run as the stable schema owner. Quote UUID-shaped service-principal names.
GRANT USAGE ON SCHEMA public TO "<app-service-principal-client-id>";
GRANT SELECT, INSERT, UPDATE, DELETE
  ON TABLE fh_submission_telemetry, fh_lineage
  TO "<app-service-principal-client-id>";
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public
  TO "<app-service-principal-client-id>";

At startup, Fabric resolves existing telemetry tables through PostgreSQL's relation catalog. This is important because the App principal's first search_path schema can differ from the schema that owns the shared tables. Fabric skips owner-only ALTER TABLE and CREATE INDEX statements when the resolved objects already exist. If a required object is genuinely absent, migration still fails rather than pretending the schema is current.

Submission telemetry sinks are failure-isolated: an MLflow or Lakebase observer error is logged but cannot terminate the App process or change an agent result. Treat missing telemetry as an operations failure anyway. The protected certification gate must prove that lifecycle events and lineage rows were actually written before promoting the release.