Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ body:
- Durable Task SDK - .NET
- Durable Task SDK - Python
- Durable Task SDK - Java
- Durable Task SDK - Go
- Durable Functions - .NET
- Durable Functions - Python
- Infrastructure / Deployment
Expand Down Expand Up @@ -53,6 +54,6 @@ body:
label: Environment
description: |
- OS: [e.g., Windows 11, macOS 14, Ubuntu 22.04]
- Runtime: [e.g., .NET 8, Python 3.11, Java 17]
- Runtime: [e.g., .NET 8, Python 3.11, Java 17, Go 1.25]
- Docker version (if using emulator)
render: markdown
3 changes: 2 additions & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,12 @@ What kind of change does this Pull Request introduce?
git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install
# Follow the sample README for language-specific setup.
```

* Test the code
<!-- Add steps to run the tests suite and/or manually test -->
<!-- For Go, run go mod download, go build ./..., go test ./..., and go vet ./... from samples/durable-task-sdks/go. State separately whether the sequential ./e2e runner was used with DTS_SAMPLES_E2E=1 and HISTORY_EXPORT_ISOLATED_TASKHUB=1 after preparing the isolated hub and Blob endpoint described in CONTRIBUTING.md. -->
```
```

Expand Down
10 changes: 10 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,16 @@ updates:
patterns: ["*"]
update-types: ["patch", "minor", "major"]

# Go (single shared module)
- package-ecosystem: "gomod"
directory: "/samples/durable-task-sdks/go"
schedule:
interval: "monthly"
groups:
gomod-all:
patterns: ["*"]
update-types: ["patch", "minor", "major"]

# JavaScript (npm)
- package-ecosystem: "npm"
directories:
Expand Down
102 changes: 102 additions & 0 deletions .github/skills/durable-task-go/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
name: durable-task-go
description: Build durable workflows in Go with the standalone Durable Task SDK and Azure Durable Task Scheduler. Use for Go orchestrations, activities, entities, timers, events, schedules, versioning, payload/history extensions, or tracing. Does not apply to Durable Functions or Microsoft Agent Framework.
---

# Durable Task Go SDK with Durable Task Scheduler

Use Go **1.25.0+** and `github.com/microsoft/durabletask-go` **v1.0.0-beta.1**. This beta SDK targets DTS; do not substitute the older `durabletask-go` storage-backend APIs or generate Azure Functions bindings.

## Start from the samples

Read the [Go sample guide](../../../samples/durable-task-sdks/go) and the relevant sample before changing code. The samples share one module; do not create a nested `go.mod`.

With the emulator already running, from the repository root:

```bash
cd samples/durable-task-sdks/go
go mod download
go run ./function-chaining
```

Each package starts its worker and client together, runs a short demonstration, prints the result, and exits. Run other samples with `go run ./<sample-name>`.

## Keep samples readable

- Limit `main.go` to the entrypoint and CLI wiring.
- Keep orchestrations, activities, client code, and worker setup in focused files in the same sample package.
- Put domain types near their behavior. Avoid catch-all utility files, unnecessary interfaces, and extra package hierarchies.
- Make the default command demonstrate the pattern, not run an exhaustive test matrix.
- Put assertions and verification helpers in `*_test.go`; provide `TestIntegration` in `integration_test.go` using `testutil.IntegrationContext(t)`.
- Keep real input validation, operational errors, and safe cleanup in application code.
- Include a short README code map and standalone Go explanations.

## Connection and lifecycle

- Default to `Endpoint=http://localhost:8080;TaskHub=default;Authentication=None`, overridden by `DTS_CONNECTION_STRING`.
- For Azure, use `Endpoint=https://<scheduler-host>;TaskHub=<hub>;Authentication=DefaultAzure` or `Authentication=AzureCLI`. The identity needs Durable Task Data Contributor access. Never hard-code real resource identifiers or credentials.
- Parse the connection string with `durabletaskscheduler.NewOptionsFromConnectionString`.
- Register orchestrators and activities with `task.NewTaskRegistry`, `AddOrchestratorN`, and `AddActivityN`; check registration errors.
- Use `durabletaskscheduler.NewClient` for management and `durabletaskscheduler.NewWorker` for execution. `client.WithAutoWorkItemFilters()` routes work by registrations, allowing sample workers to share a hub.
- Start the worker before scheduling. Use bounded client contexts, inspect terminal status and output, close the client, and shut down the worker. Clean up only resources created by the sample.
- History-export requires a dedicated hub with no other export workers and no unrelated workloads completing during its export window; automatic work-item filters do not scope history-export queries. For both emulator and Azure runs, `HISTORY_EXPORT_ISOLATED_TASKHUB=1` acknowledges verified isolation but does not create or isolate a hub. Do not validate this sample alongside other samples on a shared hub.
- Preserve the sample's [implemented ownership guards](../../../samples/durable-task-sdks/go/history-export/ownership.go) and completion-window preflight. They reject unowned pages, metadata/history reads, and Blob writes and are covered by [offline unit tests](../../../samples/durable-task-sdks/go/history-export/main_test.go). These unit checks are not evidence of emulator or live backend validation.

The [released connection guide](https://github.com/microsoft/durabletask-go/blob/v1.0.0-beta.1/durabletaskscheduler/README.md) is the authority for authentication, worker options, and feature-specific extension setup. Connecting a process to Azure is not deploying that process; these samples have no `azd` deployment templates.

## Replay-safe workflows

An orchestrator has signature `func(*task.OrchestrationContext) (any, error)`; an activity has signature `func(task.ActivityContext) (any, error)`.

- Use `ctx.CallActivity("Name", task.WithActivityInput(input)).Await(&result)` for activity results and handle errors.
- Keep HTTP, database access, filesystem access, random values, and environment reads out of orchestrators. Use activities, startup code, or supported context-bounded entity operations. External side effects can repeat if an entity operation fails before committing state.
- Use durable timers instead of `time.Sleep`; do not read the wall clock during replay.
- Use SDK tasks for concurrency, not native goroutines, channels, or `select` in orchestrators. Start all fan-out tasks before awaiting them.
- Keep ordering deterministic: sort map keys before scheduling work. Avoid mutable global state.
- Design activities for retries and possible duplicate execution; do not assume exactly-once side effects.
- Bound history with continue-as-new for recurring workflows. Sample demonstrations must still terminate.
- Use SDK entity operations and locks for durable state and coordination, not process-local mutexes. Check extension prerequisites for schedules, large payloads, and history export.
- Use numeric task versions such as `"1.0"` and `"2.0"` for DTS, even though the local registry accepts opaque version strings.

## Find the right pattern

| Need | Starting samples |
|------|------------------|
| Sequential or parallel work | [Function chaining](../../../samples/durable-task-sdks/go/function-chaining), [fan-out/fan-in](../../../samples/durable-task-sdks/go/fan-out-fan-in) |
| Wait for input or time | [Human interaction](../../../samples/durable-task-sdks/go/human-interaction), [monitoring](../../../samples/durable-task-sdks/go/monitoring) |
| Durable state | [Entities](../../../samples/durable-task-sdks/go/entities) |
| Recurring work | [Scheduled tasks](../../../samples/durable-task-sdks/go/scheduled-tasks), [bounded coordinator](../../../samples/durable-task-sdks/go/bounded-coordinator) |
| Reliability and evolution | [Saga](../../../samples/durable-task-sdks/go/saga), [versioning](../../../samples/durable-task-sdks/go/versioning), [testing](../../../samples/durable-task-sdks/go/testing) |
| Payloads and diagnostics | [Large payload](../../../samples/durable-task-sdks/go/large-payload), [history export](../../../samples/durable-task-sdks/go/history-export), [tracing](../../../samples/durable-task-sdks/go/opentelemetry-tracing) |

The [full catalog](../../../samples/README.md#go) and [pattern guide](../../../docs/patterns.md) cover the Go workflow patterns and integrations.

## Tracing

Configure an OpenTelemetry Go tracer provider/exporter and propagate the caller context when scheduling work. The Go SDK propagates W3C trace context; **DTS emits durable orchestration/activity/timer spans**. Do not claim the Go worker automatically exports those service spans locally. See the [observability guide](../../../docs/observability.md#go).

Set optional `OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318` to export application spans over OTLP/HTTP to a running collector; this does not configure DTS service-side export. The integration tests separately check trace propagation and parentage.

## Validation

Format changed Go files with `gofmt`. From `samples/durable-task-sdks/go`:

```bash
go mod download
go build ./...
go test ./...
go vet ./...
```

The beta SDK has no public in-memory testing backend. `task.Executor` is exported for internal collaboration and uses `internal/protos` parameters; do not build application tests against it. Follow the testing sample's local step adapter to exercise shared business-workflow logic offline. These unit tests do not validate SDK execution or replay.

Ordinary tests must not contact a scheduler. Replay/integration tests require explicit `DTS_SAMPLES_E2E=1` and a real DTS emulator or Azure scheduler; never claim they passed when only offline checks ran. Read [contributor guidance](../../../CONTRIBUTING.md#go-samples) before running resource-backed tests.

For full-suite validation, prepare an isolated task hub and Blob endpoint, then run demos and their compiled `TestIntegration` binaries sequentially through `./e2e` rather than enabling integration tests across all packages concurrently:

```bash
HISTORY_EXPORT_ISOLATED_TASKHUB=1 DTS_SAMPLES_E2E=1 \
go test -v -count=1 -timeout 30m ./e2e
```

Follow the [Go validation setup](../../../samples/durable-task-sdks/go/README.md#verify-every-sample-on-either-backend). The research demonstration uses synthetic fixtures by default, even with live DTS; these runs do not validate real model or arXiv services.
113 changes: 113 additions & 0 deletions .github/workflows/build-samples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,119 @@ jobs:
- name: Syntax check Python files
run: find samples -name "*.py" -not -path "*/__pycache__/*" -exec python -m py_compile {} +

go:
name: Go Samples
runs-on: ubuntu-latest
timeout-minutes: 35
defaults:
run:
shell: bash
working-directory: samples/durable-task-sdks/go
env:
DTS_SAMPLES_E2E: "0"
GO_DTS_CONTAINER: go-samples-dts-${{ github.run_id }}-${{ github.run_attempt }}
GO_AZURITE_CONTAINER: go-samples-azurite-${{ github.run_id }}-${{ github.run_attempt }}
GO_CONTAINERS_OWNER: go-samples-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: samples/durable-task-sdks/go/go.mod
cache-dependency-path: samples/durable-task-sdks/go/go.sum

- name: Download Go dependencies
run: go mod download

- name: Build Go samples
run: go build ./...

- name: Test Go samples without a scheduler
run: go test ./...

- name: Vet Go samples
run: go vet ./...

- name: Start Go integration services
timeout-minutes: 5
run: |
docker run --detach --pull=always --name "$GO_DTS_CONTAINER" \
--label "dts-samples-owner=$GO_CONTAINERS_OWNER" \
--publish 127.0.0.1:8080:8080 --publish 127.0.0.1:8082:8082 \
--env DTS_TASK_HUB_NAMES=default \
mcr.microsoft.com/dts/dts-emulator:latest
docker run --detach --pull=always --name "$GO_AZURITE_CONTAINER" \
--label "dts-samples-owner=$GO_CONTAINERS_OWNER" \
--publish 127.0.0.1:10000:10000 \
mcr.microsoft.com/azure-storage/azurite:latest \
azurite-blob --blobHost 0.0.0.0 --skipApiVersionCheck

- name: Wait for Go integration services
timeout-minutes: 3
run: |
deadline=$((SECONDS + 120))
while (( SECONDS < deadline )); do
for container in "$GO_DTS_CONTAINER" "$GO_AZURITE_CONTAINER"; do
if [[ "$(docker inspect --format '{{.State.Running}}' "$container")" != "true" ]]; then
echo "::error::Go integration container $container exited before becoming ready"
exit 1
fi
done
if curl --fail --silent --max-time 2 http://127.0.0.1:8082/ >/dev/null \
&& timeout 2s bash -c 'exec 3<>/dev/tcp/127.0.0.1/8080' 2>/dev/null \
&& timeout 2s bash -c 'exec 3<>/dev/tcp/127.0.0.1/10000' 2>/dev/null; then
echo "Go integration services are ready"
exit 0
fi
sleep 2
done
echo "::error::Go integration services did not become ready within 120 seconds"
exit 1

- name: Run Go demos and integration tests on the emulator
timeout-minutes: 17
env:
DTS_SAMPLES_E2E: "1"
HISTORY_EXPORT_ISOLATED_TASKHUB: "1"
DTS_CONNECTION_STRING: "Endpoint=http://localhost:8080;TaskHub=default;Authentication=None"
AZURE_STORAGE_CONNECTION_STRING: "UseDevelopmentStorage=true"
AZURE_STORAGE_BLOB_ENDPOINT: ""
OTEL_EXPORTER_OTLP_ENDPOINT: ""
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: ""
RESEARCH_MODE: fixture
run: go test -v -count=1 -timeout 15m ./e2e

- name: Clean up Go integration services
if: always()
timeout-minutes: 2
working-directory: ${{ github.workspace }}
run: |
containers=$(docker container ls --all --format '{{.Names}}')
cleanup_status=0
for container in "$GO_DTS_CONTAINER" "$GO_AZURITE_CONTAINER"; do
if ! grep -Fxq -- "$container" <<<"$containers"; then
continue
fi
if ! owner=$(docker inspect --format '{{ index .Config.Labels "dts-samples-owner" }}' "$container"); then
echo "::error::Could not confirm ownership of container $container"
cleanup_status=1
continue
fi
if [[ "$owner" != "$GO_CONTAINERS_OWNER" ]]; then
echo "::error::Refusing to remove container $container owned by another job"
cleanup_status=1
continue
fi
if ! docker logs --tail 100 "$container"; then
cleanup_status=1
fi
if ! docker rm --force "$container"; then
cleanup_status=1
fi
done
exit "$cleanup_status"

javascript:
name: JavaScript Samples
runs-on: ubuntu-latest
Expand Down
34 changes: 33 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,5 +80,37 @@ That's it! Thank you for your contribution!
- Each sample should have its own directory under the appropriate framework/language folder.
- Include a `README.md` following a consistent structure: description, prerequisites, how to run, and expected output.
- Use the Durable Task Scheduler emulator as the default development experience.
- Include a `requirements.txt` (Python) or project file (`.csproj`/`.sln` for .NET, `build.gradle` for Java).
- Include a `requirements.txt` (Python) or project file (`.csproj`/`.sln` for .NET, `build.gradle` for Java). Go samples share the single `go.mod` and `go.sum` in `samples/durable-task-sdks/go`; do not create nested modules.
- Test your sample with the emulator before submitting.
- Add the sample to the [catalog](./samples/README.md) and relevant [pattern documentation](./docs/patterns.md).

### Go samples

Use Go **1.25.0 or later** and the SDK version pinned in the shared module (currently `github.com/microsoft/durabletask-go` **v1.0.0-beta.1**). Put each runnable sample in its own package under `samples/durable-task-sdks/go`. The default command should start the worker and client, demonstrate the pattern, print a result, shut down, and exit.

Keep `main.go` limited to the entrypoint and CLI wiring. Put orchestrations, activities, worker setup, and client code in focused files within the same package. Keep domain types near the code that uses them; avoid catch-all utility files and unnecessary package layers. A reader should be able to understand the workflow without reading a verification harness.

Put assertions, exhaustive scenarios, and verification-only helpers in `*_test.go`. Each sample must have an opt-in `TestIntegration` in `integration_test.go`, using `testutil.IntegrationContext(t)` to select real-backend tests. Keep operational error handling, input validation, and resource cleanup in production code. README descriptions should stand alone and include a short code map.

Format changed Go files with `gofmt`, then run the same offline checks as CI:

```bash
cd samples/durable-task-sdks/go
go mod download
go build ./...
go test ./...
go vet ./...
```

Ordinary tests must not require an emulator, Azure credentials, or cloud resources. The Go beta SDK has no public in-memory testing backend: test shared business logic offline through a local step adapter, as the testing sample does. Do not claim that these unit tests validate SDK execution or replay.

Keep replay/integration tests against real DTS opt-in with `DTS_SAMPLES_E2E=1`. To run all demonstrations and their integration tests, first prepare an isolated task hub and Blob endpoint as described in the [Go validation guide](./samples/durable-task-sdks/go/README.md#verify-every-sample-on-either-backend). From the Go module, use the sequential runner rather than enabling resource-backed tests across all packages concurrently:

```bash
HISTORY_EXPORT_ISOLATED_TASKHUB=1 DTS_SAMPLES_E2E=1 \
go test -v -count=1 -timeout 30m ./e2e
```

History-export validation requires a dedicated emulator or Azure task hub with no other export workers and no unrelated workloads completing during the sample's export window. Do not run it in parallel with shared-hub sample validation. For both emulator and Azure runs, set `HISTORY_EXPORT_ISOLATED_TASKHUB=1` only after confirming isolation; the flag is an acknowledgment, not an isolation mechanism.

Use the emulator connection string by default: `Endpoint=http://localhost:8080;TaskHub=default;Authentication=None`. Document any additional prerequisites and read `DTS_CONNECTION_STRING` for Azure connections. Use placeholders, never real resource identifiers or credentials, in committed examples. See the [Go quickstart](./docs/quickstart.md#go) for connection setup.
Loading
Loading