Live tests

Environment-gated integration tests for models, sandboxes, Azure, Databricks, Cloudflare, Docker, and Temporal.

Fabric Harness keeps live provider tests opt-in. Default CI uses mocks and structural tests only; live tests require credentials and are skipped unless their FABRIC_*_TEST=1 flag is set.

Sandbox providers

Daytona

FABRIC_DAYTONA_TEST=1 \
DAYTONA_API_KEY=... \
pnpm --filter @fabric-harness/connectors test

Optional:

DAYTONA_IMAGE=ubuntu:latest

The test creates a Daytona sandbox, wraps it with daytonaSandbox(), and runs validateSandboxAdapter().

E2B

FABRIC_E2B_TEST=1 \
E2B_API_KEY=... \
pnpm --filter @fabric-harness/connectors test

Optional:

FABRIC_E2B_PACKAGE=@e2b/code-interpreter
E2B_TEMPLATE=...

Modal deployments differ by app. Provide a module that exports createModalSandboxEnv() or createSandbox() and returns a Fabric SandboxEnv:

FABRIC_MODAL_TEST=1 \
FABRIC_MODAL_ADAPTER_MODULE=file:///abs/path/modal-adapter.mjs \
pnpm --filter @fabric-harness/connectors test

Azure

Azure OpenAI

FABRIC_AZURE_OPENAI_TEST=1 \
AZURE_OPENAI_ENDPOINT=... \
AZURE_OPENAI_API_KEY=... \
AZURE_OPENAI_DEPLOYMENT=... \
pnpm --filter @fabric-harness/azure test

Foundry Agent Service

FABRIC_AZURE_FOUNDRY_TEST=1 \
AZURE_FOUNDRY_PROJECT_ENDPOINT=... \
AZURE_FOUNDRY_AGENT_ID=... \
AZURE_TOKEN=... \
pnpm --filter @fabric-harness/azure test

Optional:

FABRIC_AZURE_FOUNDRY_POLL=0

ARM tools

FABRIC_AZURE_ARM_TEST=1 \
AZURE_SUBSCRIPTION_ID=... \
AZURE_TOKEN=... \
pnpm --filter @fabric-harness/azure test

Optional resources enable individual tests:

AZURE_CONTAINER_APPS_JOB_RESOURCE_GROUP=...
AZURE_CONTAINER_APPS_JOB_NAME=...
AZURE_AKS_RESOURCE_GROUP=...
AZURE_AKS_CLUSTER_NAME=...
AZURE_AKS_TEST_COMMAND='kubectl get ns'
AZURE_ACI_RESOURCE_GROUP=...
AZURE_ACI_CONTAINER_GROUP=...
AZURE_ACI_CONTAINER_NAME=...
AZURE_ACI_TEST_COMMAND='echo fabric-harness-aci-smoke'

Databricks

FABRIC_DATABRICKS_TEST=1 \
DATABRICKS_HOST=https://dbc-...cloud.databricks.com \
DATABRICKS_TOKEN=... \
pnpm --filter @fabric-harness/databricks test

Optional resources:

DATABRICKS_WAREHOUSE_ID=...
DATABRICKS_CATALOG=main
DATABRICKS_SCHEMA=default
DATABRICKS_JOB_ID=...
FABRIC_DATABRICKS_WAIT_FOR_JOB=1
DATABRICKS_NOTEBOOK_PATH=/Repos/acme/smoke
DATABRICKS_CLUSTER_ID=...
DATABRICKS_MLFLOW_RUN_ID=...
DATABRICKS_WORKSPACE_ROOT=/Repos/acme

Existing live test families

Model providers:

FABRIC_OPENAI_TEST=1 OPENAI_API_KEY=... pnpm --filter @fabric-harness/sdk test
FABRIC_ANTHROPIC_TEST=1 ANTHROPIC_API_KEY=... pnpm --filter @fabric-harness/sdk test
FABRIC_AZURE_OPENAI_TEST=1 AZURE_OPENAI_API_KEY=... AZURE_OPENAI_ENDPOINT=... pnpm --filter @fabric-harness/sdk test

Docker sandbox:

FABRIC_DOCKER_TEST=1 pnpm --filter @fabric-harness/sdk test

Temporal:

# Local/dev-server workflow smoke. Requires a reachable Temporal frontend.
FABRIC_TEMPORAL_TEST=1 \
FABRIC_TEMPORAL_ADDRESS=localhost:7233 \
pnpm --filter @fabric-harness/temporal test -- integration.test.ts

# Real-model Temporal smoke. Requires model/provider credentials too.
FABRIC_TEMPORAL_REAL_MODEL_TEST=1 pnpm --filter @fabric-harness/temporal test -- live-model.test.ts

When using temporalio/auto-setup with Postgres in CI, set DB=postgres12, DB_PORT=5432, POSTGRES_SEEDS, POSTGRES_USER, and POSTGRES_PWD, then wait for a real Temporal gRPC connection before running tests. TCP port-open checks can pass before Temporal is ready.

Cloudflare:

FABRIC_CLOUDFLARE_TEST=1 \
FABRIC_CLOUDFLARE_WORKER_URL=... \
pnpm --filter @fabric-harness/cloudflare test

CI guidance

Keep default CI credential-free.
Run live tests in scheduled or manually triggered workflows.
Use isolated resource groups/workspaces/projects.
Prefer read-only tests unless a test is explicitly validating an execution target.
Clean up provider sandboxes with adapter cleanup: true where available.
Never echo tokens or provider SDK objects into prompts, logs, artifacts, or session history.

Live tests

On this page