Unity Catalog Agent Services

Unity Catalog Agent Services gives an externally hosted Fabric Harness agent a governed identity in Databricks. The registration appears beside tables, models, and functions in Catalog Explorer. Teams can discover it with READ_METADATA, and administrators can control access with Unity Catalog grants.

Agent Services is a Databricks Beta. During the current Beta it supports registration, discovery, metadata updates, permissions, and deletion. Databricks does not yet route runtime requests through the registered service. Use the Harness /responses endpoint to invoke the agent directly; use the Agent Service as its Unity Catalog catalog entry and permission boundary.

This page also covers the separate Supervisor Agents and managed memory Betas. Supervisor Agents are Databricks-owned agent runtimes with queryable serving endpoints; Unity Catalog Agent Services registers an externally hosted agent. Harness composes either surface without presenting the native Databricks loop as a Harness loop.

Compose a native Supervisor Agent

Supervisor Agent discovery and invocation use the official generated Databricks SDK. The lifecycle is deliberately hidden until the caller acknowledges the upstream Beta:

import {
  databricks,
  databricksManagedAgentTool,
} from '@fabric-harness/databricks';

const workspace = databricks({
  host: process.env.DATABRICKS_HOST!,
  principal: {
    kind: 'service-principal',
    host: process.env.DATABRICKS_HOST!,
    clientId: process.env.DATABRICKS_CLIENT_ID!,
    clientSecret: process.env.DATABRICKS_CLIENT_SECRET!,
  },
  supervisorAgents: { acknowledgeBeta: true },
});

const agents = await workspace.supervisorAgents!.list({ maxPages: 10 });
const tools = await workspace.supervisorAgents!.listTools(agents[0]!.supervisorAgentId!);
const answer = await workspace.supervisorAgents!.invoke(agents[0]!.supervisorAgentId!, {
  input: 'Summarize the renewal risk.',
});

const assistant = databricksManagedAgentTool(workspace.agentEndpoints.client, {
  endpointName: agents[0]!.endpointName!,
  kind: 'supervisor-agent',
  resourceId: agents[0]!.name,
  maxOutputBytes: 512_000,
});

list() and listTools() bound pagination and reject repeated page tokens. invoke() resolves the current endpoint with the official SDK, propagates cancellation, disables ambiguous invocation retries, and caps the native response admitted into Harness model context. Knowledge Assistants use the same fixed-resource tool projection with kind: 'knowledge-assistant'. The official Beta SDK is available as workspace.supervisorAgents.sdk for explicit create, update, tool, example, and delete operations. Those mutations have contract coverage only; they are not yet protected-live certified.

Use managed memory explicitly

Managed memory remains distinct from Harness session and submission persistence. An application must supply the tenant/user scope at a trusted boundary for every entry operation; the scope is not accepted from the model:

const workspace = databricks({
  host,
  principal,
  managedMemory: { acknowledgeBeta: true },
});

const memory = workspace.managedMemory!;
await memory.createEntry(
  'main.agents.revenue_memory',
  `tenant/${authenticatedTenant}/user/${authenticatedUser}`,
  {
    path: '/memories/preferences/reporting',
    contents: 'Prefer quarter-over-quarter comparisons.',
  },
);

const matches = await memory.search('main.agents.revenue_memory', {
  scope: `tenant/${authenticatedTenant}/user/${authenticatedUser}`,
  query: 'reporting preference',
  topK: 5,
});

Store and entry create/read/update/delete plus bounded search are supported behind explicit Beta acknowledgement. Paths are restricted to /memories/, parent traversal is rejected, search limits are bounded, and deletion semantics remain an application-owned retention decision. This adapter uses a reviewed narrow protocol allowlist because Databricks does not currently ship the managed memory API in its generated TypeScript SDK. It has deterministic contract tests, not retained live certification.

flowchart LR
  DEV[Developer or CI] --> SDK[databricks bundle.agentServices]
  SDK --> UC[Unity Catalog Agent Service]
  UC --> META[Catalog Explorer discovery]
  UC --> GRANTS[READ_METADATA and EXECUTE grants]
  UC --> CONN[Unity Catalog HTTP connection]
  CONN -. records host and credentials .-> APP[Fabric Harness App]
  USER[Application caller] -->|POST /responses| APP
  APP --> SESSION[Durable Harness session]
  SESSION --> DBX[Databricks data and AI services]

  classDef fabric fill:#dbeafe,stroke:#2563eb,color:#172554
  classDef governed fill:#dcfce7,stroke:#16a34a,color:#052e16
  classDef identity fill:#fef3c7,stroke:#d97706,color:#422006
  class SDK,APP,SESSION fabric
  class UC,META,GRANTS,DBX governed
  class CONN identity

The four objects involved

The workspace credential used by @fabric-harness/databricks is not sent to the external agent. The HTTP connection stores the credential used to reach the agent, such as FABRIC_HARNESS_API_TOKEN. Keeping these identities separate prevents a workspace administrator token from becoming an agent runtime secret.

Before you begin

1. Ask an account administrator to enable the Agent Services preview for the account.
2. Deploy a persistent Harness agent to a reachable HTTPS endpoint. A Databricks App built with fh build --target databricks-app exposes it at /responses.
3. Create or choose a Unity Catalog schema for agent registrations.
4. Create a Unity Catalog HTTP connection whose host points at the deployed agent.
5. Give the automation principal the required grants.

The registration principal needs:

• USE CATALOG on the parent catalog;
• USE SCHEMA and CREATE SERVICE on the parent schema;
• USE CONNECTION on the HTTP connection;
• MANAGE_ACCESS_CONTROL on the Agent Service before it manages grants.

Agent consumers typically receive READ_METADATA to discover the service and EXECUTE to express permission to use it. EXECUTE is a governance grant in this Beta; it does not create an invocation route by itself.

Install and scaffold

Install the Databricks package directly:

pnpm add @fabric-harness/databricks @fabric-harness/sdk

Or let the CLI install the package, environment template, implementation, and contract test:

fh add databricks agent-services --dry-run
fh add databricks agent-services
pnpm exec vitest run test/databricks/agent-services.test.ts

The managed recipe writes .fabricharness/databricks/agent-service.ts. Existing files are preserved unless --force is supplied, and an incompatible installed package range stops the operation before files are written.

Deploy the Harness agent

A persistent agent provides the durable conversation behind /responses:

// .fabricharness/agents/support.ts
import { createAgent } from '@fabric-harness/sdk';

export default createAgent(({ id }) => ({
  name: 'support',
  description: 'Answer governed support questions.',
  model: `databricks/${process.env.DATABRICKS_MODEL ?? 'system.ai.gpt-oss-20b'}`,
  instructions: `You are the support agent for conversation ${id}. Answer using approved support sources only.`,
  triggers: { webhook: true, manual: true },
}));

Build and deploy it as a Databricks App:

fh doctor --target databricks-app
fh build --target databricks-app
fh deploy --target databricks-app

Confirm the runtime boundary before registering it:

curl --fail --request POST "$FABRIC_AGENT_URL/responses" \
  --header "Authorization: Bearer $FABRIC_HARNESS_API_TOKEN" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "support",
    "input": "Where is my order?",
    "stream": false
  }'

See Responses API and ResponsesAgent for streaming, continuation, custom inputs, trace IDs, tenancy, and durable deployment behavior.

Create the HTTP connection

The beginner path is Catalog Explorer → Add → Create a connection:

1. Select connection type HTTP.
2. Enter the deployed agent HTTPS host.
3. Keep the connection base path at /; the Agent Service supplies /responses.
4. Select bearer-token authentication and reference a secret containing FABRIC_HARNESS_API_TOKEN.
5. Record the connection resource name returned by Databricks.

For SQL-driven infrastructure, store the token in Databricks Secrets and reference it instead of placing a credential in source code or command history:

CREATE CONNECTION fabric_support_connection TYPE HTTP
OPTIONS (
  host 'https://fabric-support-app.example.com',
  port '443',
  base_path '/',
  bearer_token secret ('fabric', 'harness_api_token')
)
COMMENT 'Credentialed connection to the Fabric support agent';

Use the resource name returned by the connection API. Current metastores can return a single-part name such as connections/fabric_support_connection; the Agent Services API also documents a scoped form such as connections/main.agents.fabric_support_connection. The Harness client accepts either shape, with or without the connections/ prefix.

For production private networking, use a Databricks-supported Private Link path where available. If the service uses IP allowlisting, allow the serverless outbound addresses used by Unity Catalog HTTP connections. The connection secures credentials but does not replace network policy.

Register the agent

Use OAuth M2M for release automation and a short-lived user OAuth profile for local administration. PATs work for local testing but should not be embedded in application configuration.

import {
  databricks,
  databricksPrincipalFromEnv,
} from '@fabric-harness/databricks';

const workspace = databricks({
  host: process.env.DATABRICKS_HOST!,
  principal: databricksPrincipalFromEnv(process.env),
  // Agent Services is a Databricks Beta; acknowledgement is intentionally explicit.
  agentServices: { acknowledgeBeta: true },
});

const services = workspace.agentServices!;

const registered = await services.create({
  catalog: 'main',
  schema: 'agents',
  id: 'support_agent',
  connection: 'fabric_support_connection',
  basePath: '/responses',
  comment: 'Governed support agent for the customer operations team',
  systemPrompt: 'Answer customer support questions using approved sources.',
});

console.log(registered.name);

create() does not silently retry. The preview API does not provide an idempotency token, and a network retry after an ambiguous response could conceal whether a registry entry was created. For reconcilers, call get() first, create on a confirmed 404, and update known fields when the entry already exists.

Discover and update registrations

const service = await services.get('main.agents.support_agent');

const page = await services.list({
  catalog: 'main',
  schema: 'agents',
});

await services.update('main.agents.support_agent', {
  comment: 'Support agent owned by Customer Operations',
  systemPrompt: 'Answer concisely from approved support sources.',
  basePath: '/responses',
});

Agent Services is a Beta endpoint that is not yet present in the modular Databricks SDK. The explicit preview client is intentionally limited to such gaps; stable services use databricksSdk().

Updates use an explicit Databricks update_mask; omitted fields remain unchanged. The current API allows updates to comment, config.system_prompt, and config.base_path. The connection is fixed at creation. To move a service to another connection, create a replacement registration and migrate grants before deleting the old one.

List the entire metastore only for administrative inventory jobs:

let pageToken: string | undefined;
do {
  const page = await services.list({ ...(pageToken ? { pageToken } : {}) });
  for (const service of page.agent_services ?? []) console.log(service.name);
  pageToken = page.next_page_token;
} while (pageToken);

Application discovery should remain schema-scoped so it does not depend on broad metastore metadata access.

Grant and revoke access

await services.grant(
  'main.agents.support_agent',
  'customer-support',
  ['EXECUTE', 'READ_METADATA'],
);

const grants = await services.permissions('main.agents.support_agent');
console.log(grants.privilege_assignments);

await services.revoke(
  'main.agents.support_agent',
  'customer-support',
  ['EXECUTE', 'READ_METADATA'],
);

Assignable privileges are EXECUTE, READ_METADATA, MANAGE, MANAGE_ACCESS_CONTROL, and ALL_PRIVILEGES. Prefer group grants, reserve MANAGE_ACCESS_CONTROL for the platform automation principal, and avoid ALL_PRIVILEGES for application callers.

Multiple changes can be applied in one permission request:

await services.updatePermissions('main.agents.support_agent', [
  { principal: 'support-users', add: ['EXECUTE', 'READ_METADATA'] },
  { principal: 'former-support-users', remove: ['EXECUTE'] },
]);

Harness validates principal names and non-empty changes before issuing a request. Unity Catalog is still the authority that decides whether the acting principal can manage the securable.

Delete safely

Deleting the Agent Service removes only its Unity Catalog registration. It does not delete or stop the external Harness App, erase its Lakebase sessions, or remove its HTTP connection.

await services.delete('main.agents.support_agent');

Use this order when retiring an agent:

1. Remove consumer grants or redirect clients.
2. Preserve required MLflow traces, audit records, and deployment receipts.
3. Delete the Agent Service registration.
4. Delete the HTTP connection only when no other governed object references it.
5. Retire the App and apply the session-retention policy.

For ephemeral CI registrations, always delete in finally:

let releaseProbeRegistered = false;
try {
  await services.create({
    catalog: 'main',
    schema: 'agents',
    id: 'fabric_release_probe',
    connection: 'fabric_support_connection',
  });
  releaseProbeRegistered = true;
  // discovery, update and permission assertions
} finally {
  if (releaseProbeRegistered) await services.delete('main.agents.fabric_release_probe');
}

Certify in every release

Fabric Harness includes a protected live check that creates a unique temporary registration, reads it, updates it, discovers it in the schema, reads its permissions, optionally exercises grant/revoke, and deletes it. Configure the databricks-live GitHub Environment:

The certification runner automatically makes agent-services required when DATABRICKS_AGENT_SERVICES_TEST=1:

pnpm --filter @fabric-harness/databricks build
node packages/databricks/dist/certify.js

Evidence is written to artifacts/databricks-certification.json with credentials redacted. A green unit suite proves request and validation contracts; only the protected workspace run proves that the preview is enabled and the actual principal has the necessary Unity Catalog grants.

Naming and tenancy

Use one schema per ownership or data-governance boundary, then give services stable names:

main.customer_operations.support_agent
main.data_platform.data_quality_agent
main.finance.close_assistant

Do not place tenant IDs or secrets in comment, systemPrompt, service names, or connection names. For a multi-tenant Harness deployment, keep tenant enforcement in the Harness principal/RBAC layer and use Unity Catalog schemas or grants for organizational boundaries. An Agent Service registration does not weaken the agent definition's Harness capability policy.

Failure guide

For the Databricks product contract and current limitations, see the official Agent Services documentation and Unity Catalog HTTP connections.