Argo Workflows

Argo Workflows orchestrates containerized tasks on Kubernetes. You describe the tasks, their dependencies, their inputs, and their outputs in Kubernetes resources. A controller turns that declaration into Pods and records what happened.

The key word is workflow. A workflow has more structure than one command. It may fan out across many inputs, wait for several branches, retry selected failures, move files between tasks, and run cleanup after completion.

If you need one independent task, start with a Kubernetes Job. If you need one independent task on a schedule, start with a CronJob. Argo Workflows earns its place when dependencies, shared data, reuse, visibility, or workflow-level controls matter.

The durable mental model

Treat Argo Workflows as a graph controller.

You submit a Workflow resource.
Its spec declares templates and an entrypoint.
The workflow controller evaluates the graph.
Kubernetes schedules Pods for runnable tasks.
Executors help capture outputs and artifacts from those Pods.
The controller updates the Workflow status as nodes finish.

A Workflow is both a definition and one execution record. Its specification states what should run. Its status stores the evolving state of that run. This dual role makes a Workflow a live object, not a reusable library entry.

Use a WorkflowTemplate for a reusable, namespace-scoped definition. Use a ClusterWorkflowTemplate only when a cluster-scoped library is justified. Submitting a template creates a separate Workflow execution.

Anatomy of a workflow

Every Workflow needs at least one template. A template defines a unit of work or a control structure. The entrypoint names the first template to execute.

Common template types include:

container for a container command
script for source code placed into a container and executed
steps for ordered groups of tasks
dag for tasks connected by dependencies
resource for an operation on a Kubernetes resource
suspend for an explicit wait point

The lowercase term template means an item under spec.templates. The resource kind WorkflowTemplate means a reusable workflow definition. Keeping that distinction prevents a common vocabulary error.

Steps and DAGs

A Steps template emphasizes stages. Outer groups run in sequence. Tasks inside one group can run in parallel. Choose Steps when the clearest explanation is “do this stage, then that stage.”

A DAG template emphasizes dependencies. A task without dependencies can start immediately. A dependent task becomes eligible after its dependency expression is satisfied. Choose a DAG when the dependency graph explains the work better than a stage list.

Both forms can express parallel work. Both can call other Steps or DAG templates. A DAG usually exposes more available parallelism because it does not impose stage boundaries that the work does not need.

Parameters, artifacts, and volumes

Parameters carry small string values. Use them for names, flags, identifiers, and small computed results. A task can consume workflow arguments or outputs from earlier tasks.

Artifacts carry files or directories. Argo loads input artifacts before the main container runs and saves output artifacts after it finishes. Most multi-node artifact flows require a configured artifact repository.

Volumes provide a filesystem mounted into Pods. A volume can fit data shared by tasks that can mount the same storage. It has different lifecycle and scheduling constraints from an artifact repository.

Choose the data mechanism by meaning:

small control value: parameter
durable file passed between tasks: artifact
shared mounted filesystem: volume
large data already owned by an external system: pass a reference when possible

Moving large data through every workflow node increases time, storage traffic, and failure exposure. Keep data where it belongs and pass identifiers when the downstream task can read it directly.

Reuse and scheduling

Start with a small Workflow while learning. Extract stable behavior into a WorkflowTemplate when multiple callers need the same contract. Define explicit inputs and outputs before you share it.

A WorkflowTemplate is namespaced. A ClusterWorkflowTemplate is available across namespaces. Cluster scope expands the sharing and permission boundary, so it also expands the review burden.

A CronWorkflow creates Workflows from a schedule. It adds workflow-aware scheduling controls, including concurrency policy, suspension, missed-start handling, and history limits. It does not turn non-idempotent work into safe scheduled work. Design every scheduled workflow for the possibility of delayed or repeated execution.

Argo Events can submit a Workflow after an event. Argo Events owns event receipt and dependency resolution. Argo Workflows owns the resulting task execution. Keep that handoff visible when you troubleshoot.

Failure is part of the graph

Set retries for failures that may succeed on another attempt. Bound them with a limit and choose a suitable retry policy and backoff. A deterministic bad input will not improve because you retry it five times.

Set workflow and template timeouts so abandoned work does not run forever. Use exit handlers for workflow-wide cleanup or notification after the main entrypoint completes. Use lifecycle hooks when an action must react to an expression during a workflow or template lifecycle.

Retries, timeouts, and hooks do not replace idempotency. A task may fail after changing an external system but before reporting success. Design side effects so another attempt can detect or safely repeat the change.

Control concurrency at the narrowest useful boundary. Workflow and template parallelism fields limit runnable work within a graph. Controller limits bound active workflows across the installation. Mutexes and semaphores protect scarce or exclusive resources.

Components and responsibility

The workflow controller performs reconciliation. It watches Workflow resources, decides which nodes can run, creates or observes related Kubernetes resources, and updates status.

Argo Server exposes the API and web interface. It is useful for remote clients and human operations, but the controller can operate without it. Authentication to Argo Server and authorization in Kubernetes are separate controls.

Task Pods run the workload. Their workflow executor supports output and artifact handling. Kubernetes remains responsible for Pod scheduling, container runtime integration, volumes, and node placement.

This division matters during incidents. A pending Workflow node can indicate a dependency, synchronization, or controller constraint. A pending Pod usually points toward Kubernetes scheduling or admission. A running task that cannot reach storage points toward the workload, network, credentials, or artifact configuration.

Security boundaries

A workflow can start containers and may create Kubernetes resources. Permission to submit an unrestricted workflow is therefore a code-execution boundary.

Run workflows with explicit service accounts. Bind only the permissions each workload needs. Separate controller permissions, server permissions, and workflow permissions. Do not grant a workflow broad rights because one template needs one Kubernetes operation.

Restrict who can create Workflows and reusable templates. Protect artifact credentials and application secrets. Apply Pod security controls, network policy, resource quotas, and image policy as you would for other untrusted or multitenant workloads.

Argo Server needs transport security, an authentication mode, and appropriate Kubernetes authorization. A login to the interface does not by itself make every Kubernetes action safe.

Operations and scale

Observe both the Workflow resource and the Pods it creates. The graph view explains dependency state. Pod events and logs explain workload execution. Controller logs and metrics explain reconciliation queues, errors, and saturation.

Completed Workflow objects consume Kubernetes API storage. Configure time-to-live cleanup for live resources. Use the workflow archive when you need long-term execution records in a supported database. The archive stores workflow status and node results, but it does not archive Pod logs. Send logs to a logging system or configured archive path.

Large Workflow status objects can pressure the Kubernetes API. Argo can offload node status to a database. This is an operational scaling feature, not a substitute for smaller graphs or sensible retention.

Pin controller and CLI versions according to the release guidance. Read the upgrading guide before changing custom resource definitions or controller versions. Test stored templates and representative workflows against the target version.

Where Argo Workflows fits

Good fits include data processing, machine learning pipelines, batch processing, infrastructure automation, and container-based integration pipelines. These uses share a graph of bounded tasks and benefit from Kubernetes scheduling.

Argo Workflows is a poor fit when Kubernetes itself is unwanted, one Job covers the requirement, or the work needs a long-lived service rather than a finite run. It also does not provide domain-specific data lineage, model governance, or business-process semantics by itself.

The tool orchestrates work. It does not make task code correct, side effects reversible, storage durable, or credentials safe. Those remain design responsibilities.

A path from first run to production

Run one container Workflow and inspect its status and Pod.
Build the same small graph once with Steps and once with a DAG.
Pass one parameter and one artifact between tasks.
Extract a stable definition into a WorkflowTemplate.
Add a bounded retry, timeout, and exit handler.
Run with a least-privilege service account.
Add concurrency limits and test failure during a side effect.
Configure retention, metrics, logs, and archive behavior.
Test upgrades and disaster recovery with representative workflows.

You reach competence by reading the graph and the Kubernetes resources together. You reach operational confidence by forcing failures and confirming who owns each recovery step.