RAG on Databricks

Fabric Harness wires Databricks-native products for retrieval, generation, tracing, and evaluation. It does not reimplement Vector Search, Unity AI Gateway, the offline index pipeline, or MLflow judges.

Follow the same mental model as the Databricks AI Cookbook RAG inference chain:

1. (Optional) preprocess the user query
2. Retrieve with Mosaic Vector Search
3. Augment the prompt with retrieved context
4. Generate through Unity AI Gateway or a custom Model Serving endpoint
5. Validate inline citations and apply answer limits

Offline chunk → embed → index remains a Databricks Job / notebook / Lakeflow concern. Quality measurement uses MLflow 3 evaluation and managed Databricks judges, not a parallel Fabric-only judge product.

Quick start (managed recipe)

fh add databricks rag-chain
# set DATABRICKS_HOST, OAuth credentials, DATABRICKS_VECTOR_INDEX, DATABRICKS_MODEL
fh run rag-answer --question "How do I reset my password?"

Scaffolded files:

Related recipes:

Code API

Deterministic chain (cookbook online path)

import { databricksRagChain, toMlflow3EvaluationRecord, exportMlflow3EvaluationJsonl } from '@fabric-harness/databricks';

const chain = databricksRagChain({
  databricks: {
    host: process.env.DATABRICKS_HOST!,
    principal: { kind: 'pat', token: process.env.DATABRICKS_TOKEN! },
    model: 'system.ai.gpt-oss-20b',
    vectorSearch: {
      index: 'main.support.kb_index',
      textColumn: 'chunk',
      idColumn: 'id',
    },
  },
  topK: 5,
  postProcess: {
    requireCitations: true,
    citationRepairAttempts: 2,
  },
});

const turn = await chain.invoke({ question: 'How do I reset my password?' });
// turn.answer, turn.sources (retrieved), turn.citations (actually referenced), turn.usage

With requireCitations, Fabric asks the model to revise an uncited answer using only the retrieved source IDs. Repair is bounded and fails closed if the model still omits a valid marker; Fabric never adds a citation to the answer on the model's behalf.

What runs under the hood:

Agentic multi-tool RAG

When the agent must also call SQL, Genie, or Jobs, keep tool-calling:

const chain = databricksRagChain({ databricks: { /* + vectorSearch */ } });
const { modelProvider, tools, policy } = chain.asAgentTools();
// pass into init({ modelProvider, tools, policy })

Or continue using databricks({ vectorSearch }) + bundle.tools as in examples/with-databricks-rag.

Evaluation and quality (native Databricks first)

Local CI smoke (optional)

Lightweight checks on a RagTurn validate actual [source-id] markers, required facts, and retrieval:

import { scoreRagTurn, toMlflow3EvaluationRecord, exportMlflow3EvaluationJsonl } from '@fabric-harness/databricks';

const scores = scoreRagTurn(turn, {
  mustContain: ['Settings'],
  requireCitations: true,
  mustRetrieveIds: ['doc-1'],
});

These are smoke scorers, not a substitute for managed evaluation.

MLflow 3 managed evaluation

Export MLflow 3 rows with structured inputs, outputs, expectations, retrieved context, and trace metadata:

const record = toMlflow3EvaluationRecord(turn, {
  expectedAnswer: 'Use Settings, then Security.',
  traceId: 'tr-...',
  submissionId: 'sub-...',
});
const jsonl = exportMlflow3EvaluationJsonl([record]);
// Merge the record into an MLflow Evaluation Dataset from a Databricks notebook or job.

The release certification fixture runs this loop as a serverless Databricks Job. The notebook at scripts/databricks/rag_evaluation.py queries five candidates from the real Vector Search index, uses the configured Databricks model to retain only sources that supply relevant or complementary evidence, records the filtered documents in the retriever span, and generates a citation-backed answer. It merges the governed 50-case golden set into a content-hashed Unity Catalog evaluation dataset, so changed fixtures cannot inherit stale rows from an earlier release, and runs the following managed judges:

The source documents and cases live together in scripts/databricks/rag_fixture.json. The provisioner merges those documents into the Delta source table, triggers the Vector Search index, and embeds the same cases into the evaluation notebook. This prevents the index fixture and golden set from drifting apart.

Each aggregate must meet DATABRICKS_RAG_EVAL_THRESHOLD (default 0.8). Evidence includes the MLflow run, versioned dataset and fixture hash, generation and judge models, category counts, selected aggregate metrics, and threshold result. A failed judge fails the Databricks Job and therefore the release certification gate.

The protected release workflow for commit 5fcf927bcf0a3a15e3aec86e2629516cbb76ad26 evaluated fixture 18bd4dffc32c as MLflow run 18131356bcac4460a26de20d40b9573a in the reference Azure workspace. It covered all 50 cases with databricks-gpt-oss-120b for generation and managed judging, and passed the unchanged 0.8 floor:

This is retained workspace evidence for the reference fixture, not a claim that every application or dataset will achieve the same scores. Replace the documents, question-level expected facts, and adversarial cases with your domain corpus before treating the gate as production evidence.

Provision or repair the disposable fixture, then run the same gate used by protected CI:

FABRIC_DATABRICKS_PROVISION=1 pnpm databricks:cert:provision
pnpm databricks:certify

The provisioner is idempotent: it reuses the UC dataset, Jobs, Vector Search index, Feature Serving endpoint, Genie space, and Lakeflow pipeline. Improve quality by diagnosing retrieval versus generation, changing topK, prompts, chunks, or embeddings, and rerunning the managed judges before redeploying Apps or Model Serving.

The default job limits MLflow to one prediction worker and one scorer worker, with prediction and scorer rate limits, so pay-per-token endpoints do not exceed workspace output-token quotas. Raise MLFLOW_GENAI_EVAL_MAX_WORKERS, MLFLOW_GENAI_EVAL_MAX_SCORER_WORKERS, MLFLOW_GENAI_EVAL_PREDICT_RATE_LIMIT, or MLFLOW_GENAI_EVAL_SCORER_RATE_LIMIT only when the workspace uses capacity that supports the additional concurrency.

Do not build a second full judge product in Fabric when MLflow managed evaluation already exists in the workspace.

Offline index pipeline (not the chain)

Building/updating the Vector Search index (chunking, embedding, write) stays on Databricks:

• Notebooks / Jobs
• Lakeflow pipelines (fh add lakeflow)
• UC Volumes as document sources

Fabric agents consume the index via Vector Search at inference time.

Guardrails

The default chain treats retrieved chunks as untrusted data, serializes them as bounded JSONL, limits per-chunk and total context size, and rejects citation markers that do not match a retrieved source. turn.sources means "retrieved"; only source IDs referenced by the final answer appear in turn.citations.

const chain = databricksRagChain({
  databricks: { /* model + vectorSearch */ },
  topK: 5,
  maxContextChars: 32_000,
  maxChunkChars: 8_000,
  postProcess: {
    citationPolicy: 'validate',
    maxAnswerChars: 8_000,
  },
});

See also

• Databricks recipes
• Integrations map
• example: with-databricks-rag
• Cookbook: RAG chain for inference