Responses API and ResponsesAgent

Fabric Harness exposes persistent agents through POST /responses, the interface Databricks uses for new agent Apps. The same contract is packaged as an MLflow ResponsesAgent for Model Serving, AI Playground, evaluation, monitoring, and clients that use DatabricksOpenAI.responses.

flowchart LR
  CLIENT[DatabricksOpenAI or HTTP client] -->|Direct POST /responses| APP[Databricks App]
  CLIENT -->|Serving invocation| SERVING[Model Serving and AI Playground]
  SERVING --> PROXY[MLflow ResponsesAgent proxy]
  PROXY -->|POST /responses| APP
  APP --> AUTH[OAuth or App OBO identity]
  AUTH --> API[Fabric Responses adapter]
  API --> SUB[Durable submission]
  SUB --> AGENT[Persistent Fabric agent]
  AGENT --> DBX[AI Gateway, SQL, AI Search, Genie, MCP]
  AGENT --> EVENTS[Token and tool events]
  EVENTS -->|SSE| CLIENT
  SUB --> STORE[Lakebase session and submission stores]
  SUB --> TRACE[MLflow trace]

Build your first Responses agent

Create .fabricharness/agents/analyst.ts:

import { createAgent } from '@fabric-harness/sdk';

export default createAgent(({ id }) => ({
  name: 'analyst',
  model: 'databricks/system.ai.gpt-oss-20b',
  sandbox: 'virtual',
  instructions: `You are a governed data analyst for instance ${id}.`,
  triggers: { webhook: true },
}));

Enable the endpoint in .fabricharness/config.ts:

import type { FabricHarnessConfig } from '@fabric-harness/node';

export default {
  run: { idPrefix: 'analytics-agent' },
  responses: {
    agent: 'analyst',
    defaultSession: 'default',
    customOutputKeys: ['client_type'],
  },
} satisfies FabricHarnessConfig;

custom_inputs are not added to model context. customOutputKeys is an explicit allowlist for non-secret values that may be returned under custom_outputs; all other custom fields stay outside the prompt and response.

By default, each authenticated principal gets a separate persistent agent instance. When you set responses.instanceId, it acts as a namespace prefix and Harness still appends the principal id. This prevents two App users who choose the same conversation id from sharing session history.

Start locally:

pnpm add @fabric-harness/sdk @fabric-harness/node @fabric-harness/databricks
pnpm exec fh dev

Call it without streaming:

curl http://localhost:3000/responses \
  -H 'content-type: application/json' \
  -H 'idempotency-key: analyst-demo-1' \
  -d '{
    "input": [{"role":"user","content":"Summarize yesterday orders"}],
    "context": {"conversation_id":"orders-42"},
    "custom_inputs": {"client_type":"local-demo"}
  }'

Set "stream": true to receive response.output_text.delta events followed by one response.output_item.done event. All events for an answer use the same item id. Harness normalizes both ordinary string deltas and the typed reasoning/output blocks returned by Unity AI Gateway. Reasoning blocks remain internal; typed text and output_text blocks become incremental response deltas and the completed output item.

Continue a conversation

Use either durable conversation mechanism:

• Send the same context.conversation_id for each turn. The authenticated principal selects the persistent agent instance, so one caller cannot select another caller's instance.
• Send the prior response id as previous_response_id. Fabric resolves the original durable submission and reuses its instance and session after checking tenant and principal ownership.

An Idempotency-Key is scoped to tenant, principal, agent, and key. Retrying the same request returns the same response id. Reusing the key with a different payload is rejected by the submission store. Treat aggregate and streaming calls as separate invocation modes: retry either mode with its original key, but use a new key when changing stream because that changes the admitted request payload.

Authentication on Databricks

Locally, use the server bearer token or your configured OIDC authenticator. In a Databricks App:

• App authorization runs as the App service principal.
• User authorization validates the forwarded token against the workspace current-user API before accepting it. Validation is cached by token digest and bounded by token expiry.
• Each validated App user receives a stable, opaque Fabric tenant by default. Session listing, direct reads, approvals, artifacts, abort, and deletion remain inside that tenant; legacy unscoped sessions are not visible to a tenant-bound user.
• App users receive only invoke and own-session permissions. The generated authenticator does not grant admin:read, build:read, session:replay, or wildcard access.
• Clients must call Apps with Databricks OAuth. Personal access tokens are not supported for App URLs.

The generated Declarative Automation Bundle requests the sql, genie, and model-serving user API scopes. Databricks combines those scopes with each user's existing workspace and Unity Catalog permissions. Users must consent after scopes change; restart an older App before adding scopes if the workspace requires it.

The App build automatically exposes the first persistent agent. Set responses.agent when the workspace contains multiple persistent agents.

fh build --target databricks-app
fh deploy --target databricks-app

TOKEN="$(databricks auth token --host "$DATABRICKS_HOST" | jq -r .access_token)"
curl "$DATABRICKS_APP_URL/responses" \
  -H "Authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -H 'x-mlflow-return-trace-id: true' \
  -d '{"input":[{"role":"user","content":"What changed in revenue?"}],"stream":true}'

When DATABRICKS_MLFLOW_EXPERIMENT_ID configures the App's MLflow exporter and x-mlflow-return-trace-id: true is present, a non-streaming response includes metadata.trace_id; a stream emits a separate trace-id event before [DONE]. The identifier matches the deterministic MLflow trace produced for the durable submission.

GET /ready reports mlflowTracing: configured without returning the experiment identifier. The bundle attaches the experiment with CAN_EDIT, and the final app.yaml resolves that managed resource through valueFrom. A Databricks App build is not ready when that value was not injected.

Model Serving interoperability

Build the wrapper after the App is reachable:

pnpm add @fabric-harness/databricks
fh build --target databricks-serving
cd .fabricharness/build/databricks-serving
python serving/contract_test.py
python3 -m pip install -r serving/requirements.txt
fh deploy --target databricks-serving

The CLI loads the native Databricks SDK only for the Serving deployment operation. The integration package is therefore an optional CLI peer and an explicit project dependency; a missing package fails before model registration with an install instruction. Databricks init templates and managed recipes add it automatically.

The artifact subclasses mlflow.pyfunc.ResponsesAgent, declares the MLflow task agent/v1/responses, forwards non-streaming and streaming requests to the App, preserves custom_inputs, and authenticates to a Databricks App with short-lived OAuth M2M credentials. Use an App-dedicated service principal with CAN USE; do not put a personal access token or long-lived workspace token in Model Serving environment variables.

The generated requirements accept MLflow >=3.10; the protected Azure release gate currently validates model logging and deployment with MLflow 3.14.0. Optional request fields are read with attribute-safe fallbacks because MLflow's own model-validation input may omit databricks_options. Run serving/contract_test.py before logging the model and retain the live model-registration result as the compatibility record for your workspace.

After model registration, fh deploy uses the typed TypeScript serving-admin client to create or update the endpoint and enable an AI Gateway inference table. Payload logging is written under the configured serving catalog and schema with a table prefix derived from the endpoint name. The CLI serializes these mutations: it waits for endpoint readiness before replacing an existing model, waits again before the AI Gateway update, and waits once more before reporting success. This makes release retries safe when an earlier endpoint or gateway configuration is still converging. Databricks agent endpoints currently support inference tables but not Gateway rate limits; the App's authenticated HTTP limiter remains the enforced request boundary.

Request reference

Production errors do not return upstream bodies or secrets. Aborted responses return 409 in non-streaming mode; streams emit response.failed with a bounded Databricks error object.

Release verification

Every change runs the Node request/parser and durable endpoint tests plus the generated Python contract. The protected Databricks workflow deploys the App and MLflow wrapper, then verifies OAuth, non-streaming output, streaming deltas, stable item ids, trace correlation, restart recovery, and secret redaction. The public capability registry links the retained 4.4.1 Azure evidence and marks this surface protected-live; its product status remains Beta because the upstream API is Beta and it is still a Tier O, Serving-profile check rather than a core Tier R release requirement. See compatibility and certification for the exact artifact identity.

References: Databricks agent authoring, query deployed agents, and MLflow ResponsesAgent.