Cadence

Migration Notes

• Cassandra schema: transfer task and timer task tables require new columns for domain multi-tenancy task routing. Apply schema migrations before rolling out.
• Histogram metrics: histograms config block must be explicitly set to opt into histogram emission. See Timer to Histogram Metric Migration section above for the full rollout path.

Major Features

1. Cadence Schedules

Cadence Schedules is now generally available. See the blog post and concepts documentation. Introduces a first-class scheduling primitive that replaces hand-rolled cron workflows. The full API surface (Create, Describe, Update, Delete, Pause, Unpause, List, Backfill) is implemented across the type system, proto mappers, Thrift transport, frontend handlers, and CLI. The scheduler runs as a per-domain durable workflow with ContinueAsNew, enforcing configurable overlap policies (SKIP, CONCURRENT, BUFFER, CATCH_UP_ALL), catch-up for missed fire times, concurrency limits, backfill with per-fire override policies, and search attribute propagation to target workflows. A per-domain scheduler worker manager uses membership-based routing and is enabled by default with a per-domain filter.

Migration Path

• The scheduler worker is disabled by default. Operators can enable it per-domain via dynamic config.
• Schedule search attributes must be added to your visibility index. They are included in the default key set automatically.

2. Active-Active Domains

Features

Added MySQL support for Active-Active

Cadence Operations Failover workflow now supports passing the set of cluster-attributes that should be failed over

Bug Fixes

It is no longer possible to convert a domain to active-active via the FailoverDomain RPC

3. Domain Multi-Tenancy (in progress)

Extends Cadence's multi-tenancy guarantees below the domain level. Cadence already enforces rate limiting, task prioritization, and fair task scheduling at the domain level to prevent noisy-neighbor interference between customers. However, platform teams that consolidate multiple use cases into a single domain have historically had no isolation between those use cases. This release introduces task-list-level isolation primitives, including task routing, rate limiting, and scheduling priority, so that platform teams can get the same guarantees within a domain that domain-level isolation provides between domains.

Key Capabilities

• Hierarchical weighted round-robin (IWRR) scheduler for fair task scheduling across task lists within a domain
• Task list nice value for priority control between task lists within a domain
• Task list rate limiting applied to both regular and sticky polls, preventing a single task list from starving others in the same domain
• Task list name and kind stored in history and transfer tasks to enable task routing decisions
• History task latency metrics tracked at the task list level for per-task-list observability

Migration Path

• Cassandra schema upgrade required: transfer task and timer task tables gain new columns for task list name and kind. Apply schema migrations before rolling out this version.

Current Limitations

Tasklist-level isolation is currently in active development. Additional controls and enforcement points will be added in subsequent releases.

4. Timer to Histogram Metric Migration

Cadence is migrating from Tally's costly Timer metrics to ExponentialHistogram to reduce infrastructure costs and align with Prometheus and OpenTelemetry standards.

The v1.4.1 release enables dual-emission for all timer metri

Cadence

Build Status

Coverage

Slack Status

Github release

License

Cadence Workflow is an open-source platform since 2017 for building and running scalable, fault-tolerant, and long-running workflows. This repository contains the core orchestration engine and tools including CLI, schema managment, benchmark and canary.

Getting Started

Cadence backend consists of multiple services, a database (Cassandra/MySQL/PostgreSQL) and optionally Kafka+Elasticsearch.

As a user, you need a worker which contains your workflow implementation.

Once you have Cadence backend and worker(s) running, you can trigger workflows by using SDKs or via CLI.

1. Start cadence backend components locally

docker compose -f docker/docker-compose.yml up

1. Run the Samples

Try out the sample recipes for Go or Java.

1. Visit UI

Visit http://localhost:8088 to check workflow histories and detailed traces.

Kubernetes Deployment

For a guided Kubernetes installation experience, KubeStellar Console provides a step-by-step mission that deploys Cadence using the official Helm chart from cadence-charts. The mission includes pre-flight checks, validation, troubleshooting, and rollback support.

Client Libraries

You can implement your workflows with one of our client libraries:

• Official Cadence Go SDK
• Official Cadence Java SDK

There are also unofficial Python and Ruby SDKs developed by the community.

You can also use iWF as a DSL framework on top of Cadence.

CLI

Cadence CLI can be used to operate workflows, tasklist, domain and even the clusters.

You can use the following ways to install Cadence CLI:

• Use brew to install CLI: brew install cadence-workflow
• Follow the instructions if you need to install older versions of CLI via homebrew. Usually this is only needed when you are running a server of a too old version.
• Use docker image for CLI: docker run --rm ubercadence/cli:<releaseVersion> or docker run --rm ubercadence/cli:master . Be sure to update your image when you want to try new features: docker pull ubercadence/cli:master
• Build the CLI binary yourself, check out the repo and run make cadence to build all tools. See CONTRIBUTING for prerequisite of make command.
• Build the CLI image yourself, see instructions

Cadence CLI is a powerful tool. The commands are organized by tabs. E.g. workflow->batch->start, or admin->workflow->describe.

Please read the documentation and always try out --help on any tab to learn & explore.

UI

Try out Cadence Web UI to view your workflows on Cadence.

(This is already available at localhost:8088 if you run Cadence with docker compose)

Other binaries in this repo

Bench/stress test workflow tools

See bench documentation.

Periodical feature health check workflow tools(aka Canary)

See canary documentation.

Schema tools for SQL and Cassandra

The tools are for manual setup or upgrading database schema

• If server runs with Cassandra, Use Cadence Cassandra tool
• If server runs with SQL database, Use Cadence SQL tool

The easiest way to get the schema tool is via homebrew.

brew install cadence-workflow also includes cadence-sql-tool and cadence-cassandra-tool.

• The schema files are located at /usr/local/etc/cadence/schema/.
• To upgrade, make sure you remove the old ElasticSearch schema first: mv /usr/local/etc/cadence/schema/elasticsearch /usr/local/etc/cadence/schema/elasticsearch.old && brew upgrade cadence-workflow. Otherwise ElasticSearch schemas may not be able to get updated.
• Follow the instructions if you need to install older versions of schema tools via homebrew.

However, easier way is to use new versions of schema tools with old versions of schemas.

All you need is to check out the older version of schemas from this repo. Run git checkout v0.21.3 to get the v0.21.3 schemas in the schema folder.

Contributing

We'd love your help in making Cadence great. Please review our contribution guide.

If you'd like to propose a new feature, first join the CNCF Slack workspace in the #cadence-users channel to start a discussion.

Please visit our documentation site for production/cluster setup.

Learning Resources

See Maxim's talk at Data@Scale Conference for an architectural overview of Cadence.

Visit cadenceworkflow.io to learn more about Cadence. Join us in Cadence Documentation project. Feel free to raise an Issue or Pull Request there.

Community

• Github Discussion
• Best for Q&A, support/help, general discusion, and annoucement
• Github Issues
• Best for reporting bugs and feature requests
• StackOverflow
• Best for Q&A and general discusion
• Slack - Join #cadence-users channel on CNCF Slack
• Best for contributing/development discussion

Stars over time

Stargazers over time

License

Apache 2.0 License, please see LICENSE for details.