Argo Workflows
Argo Workflows is a container-native workflow engine for Kubernetes that orchestrates parallel jobs as directed acyclic graphs. Each step runs in its own container, making it suitable for CI/CD pipelines, data processing, and machine learning workflows.
itCloud native tools and technologies | OpenSkills.info
Intro
Argo Workflows
Argo Workflows orchestrates containerized tasks on Kubernetes. You describe the tasks, their dependencies, their inputs, and their outputs in Kubernetes resources. A controller turns that declaration into Pods and records what happened.
The key word is workflow. A workflow has more structure than one command. It may fan out across many inputs, wait for several branches, retry selected failures, move files between tasks, and run cleanup after completion.
If you need one independent task, start with a Kubernetes Job. If you need one independent task on a schedule, start with a CronJob. Argo Workflows earns its place when dependencies, shared data, reuse, visibility, or workflow-level controls matter.
The durable mental model
Treat Argo Workflows as a graph controller.
- You submit a
Workflowresource. - Its
specdeclares templates and an entrypoint. - The workflow controller evaluates the graph.
- Kubernetes schedules Pods for runnable tasks.
- Executors help capture outputs and artifacts from those Pods.
- The controller updates the
Workflowstatus as nodes finish.
A Workflow is both a definition and one execution record. Its specification states what should run. Its status stores the evolving state of that run. This dual role makes a Workflow a live object, not a reusable library entry.
Use a WorkflowTemplate for a reusable, namespace-scoped definition. Use a ClusterWorkflowTemplate only when a cluster-scoped library is justified. Submitting a template creates a separate Workflow execution.
Anatomy of a workflow
Every Workflow needs at least one template. A template defines a unit of work or a control structure. The entrypoint names the first template to execute.
Common template types include:
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://argo-workflows.readthedocs.io/en/latest/
Supports
- Argo Workflows as a container-native workflow engine for parallel jobs on Kubernetes
- Workflow implementation through Kubernetes custom resources
- Steps, DAG, artifact, scheduling, interface, retry, timeout, hook, metrics, and archive feature map
- Machine learning, data processing, infrastructure automation, and CI or CD use cases
- https://github.com/argoproj/argo-workflows/releases
Supports
- Argo Workflows v4.0.5 as the current release snapshot used for this course
- Versioned controller, server, and CLI release artifacts
- Release notes and upgrade guidance as inputs to version changes
- https://argo-workflows.readthedocs.io/en/latest/quick-start/
Supports
- Current quick-start installation prerequisites and warnings
- Submission, listing, inspection, log viewing, and web interface flow for a first Workflow
- Quick-start scope as a learning environment rather than a production installation
- https://argo-workflows.readthedocs.io/en/latest/walk-through/
Supports
- Official sequence from CLI and first Workflow through parameters, graph forms, data, control flow, retries, timeouts, and handlers
- Foundation-to-intermediate practice path used by the course links
- https://argo-workflows.readthedocs.io/en/latest/workflow-concepts/
Supports
- Workflow as both an executable definition and an execution-state record
- Workflow specification structure with templates and an entrypoint
- Container, script, resource, suspend, Steps, and DAG template roles
- Difference between task templates and reusable workflow resources
- https://argo-workflows.readthedocs.io/en/latest/architecture/
Supports
- Workflow controller responsibility for reconciliation
- Argo Server responsibility for the API and web interface
- Controller operation without a required Argo Server
- Workflow executor role in task execution, outputs, and artifacts
- https://argo-workflows.readthedocs.io/en/latest/walk-through/steps/
Supports
- Sequential outer step groups and parallel tasks within one group
- Steps as a stage-oriented workflow structure
- https://argo-workflows.readthedocs.io/en/latest/walk-through/dag/
Supports
- DAG tasks with explicit dependencies
- Immediate eligibility of root tasks and parallel execution of independent branches
- Nested Steps and DAG templates
- DAG fail-fast behavior and dependency expressions
- https://argo-workflows.readthedocs.io/en/latest/walk-through/parameters/
Supports
- Workflow and template input parameters
- Parameter passing through arguments and output references
- Parameters as small values in task contracts
- https://argo-workflows.readthedocs.io/en/latest/walk-through/artifacts/
Supports
- Input and output artifacts as files or directories
- Executor handling before and after the main container
- Artifact repositories for passing artifacts between tasks
- Difference between artifact paths and parameter values
- https://argo-workflows.readthedocs.io/en/latest/workflow-templates/
Supports
- WorkflowTemplate as a namespace-scoped reusable definition
- Submission and reference of reusable templates
- Difference between lowercase template and WorkflowTemplate resource
- Explicit parameter passing into referenced templates
- https://argo-workflows.readthedocs.io/en/latest/cluster-workflow-templates/
Supports
- ClusterWorkflowTemplate as a cluster-scoped reusable definition
- Cross-namespace availability and cluster-scope references
- Cluster scope as a wider authorization and change boundary
- https://argo-workflows.readthedocs.io/en/latest/cron-workflows/
Supports
- CronWorkflow creation of Workflows from schedules
- Time zone, suspension, concurrency policy, missed-start, and history controls
- Need to account for duplicate or delayed scheduled execution
- https://argoproj.github.io/argo-events/sensors/triggers/argo-workflow/
Supports
- Argo Events Sensor submission of Argo Workflow resources
- Boundary between event dependency resolution and downstream workflow execution
- https://argo-workflows.readthedocs.io/en/latest/retries/
Supports
- Retry limits, policies, expressions, and backoff
- Difference between failed and errored task outcomes
- Operational need to bound repeated attempts
- https://argo-workflows.readthedocs.io/en/latest/walk-through/exit-handlers/
Supports
- Workflow exit handler execution after the entrypoint completes
- Workflow status availability to final cleanup and notification logic
- Difference between final handling and ordinary graph tasks
- https://argo-workflows.readthedocs.io/en/latest/lifecyclehook/
Supports
- Lifecycle hooks triggered from workflow or template expressions
- Difference between lifecycle hooks and exit handlers
- https://argo-workflows.readthedocs.io/en/latest/synchronization/
Supports
- Mutexes for exclusive execution
- Semaphores for bounded concurrent execution
- Workflow and template parallelism controls
- Workflow status as evidence of lock holding and waiting
- https://argo-workflows.readthedocs.io/en/latest/parallelism/
Supports
- Controller-wide and namespace workflow limits
- Workflow priority under controller limits
- Relationship between parallelism limits and synchronization
- https://argo-workflows.readthedocs.io/en/latest/service-accounts/
Supports
- Workflow service accounts as Kubernetes API identities
- Role and RoleBinding use for workload permissions
- Default service account behavior when no explicit account is supplied
- https://argo-workflows.readthedocs.io/en/latest/workflow-rbac/
Supports
- Workflow Pod permissions through Kubernetes service accounts
- Least-privilege RBAC for executor and resource operations
- Separation of controller and workflow workload permissions
- https://argo-workflows.readthedocs.io/en/latest/security/
Supports
- Security boundaries across controller, server, workflow Pods, secrets, and networks
- Authentication, transport security, and authorization requirements for Argo Server
- Risk from untrusted workflow specifications and container execution
- Multitenancy controls and separation of workload identities
- https://argo-workflows.readthedocs.io/en/latest/argo-server/
Supports
- Argo Server API and web interface responsibilities
- Authentication modes, transport security, and exposure considerations
- Distinction between server access and Kubernetes authorization
- https://argo-workflows.readthedocs.io/en/latest/workflow-archive/
Supports
- Database-backed retention of completed workflow status and node results
- Exclusion of Pod logs from the workflow archive
- Separate archive-log or logging-system requirement
- https://argo-workflows.readthedocs.io/en/latest/offloading-large-workflows/
Supports
- Kubernetes object size pressure from large Workflow node status
- Database offloading of node status for large workflows
- Persistence dependency introduced by offloading
- https://argo-workflows.readthedocs.io/en/latest/metrics/
Supports
- Controller queue, workflow, and custom metrics
- Metrics as current process and workflow signals rather than per-run historical storage
- Workflow archive and logs as sources for historical execution analysis
- https://argo-workflows.readthedocs.io/en/latest/running-at-massive-scale/
Supports
- Controller sizing and scaling considerations
- API pressure, parallelism, status offloading, and retention controls
- Need to test representative workflow shapes under expected load
- https://argo-workflows.readthedocs.io/en/latest/upgrading/
Supports
- Version-specific upgrade and breaking-change guidance
- Need to review controller, custom resource definition, server, and client changes
- Testing before production version changes
- https://kubernetes.io/docs/concepts/workloads/controllers/job/
Supports
- Kubernetes Job as the built-in controller for finite tasks
- Job management of Pods toward successful completion
- Job as the smaller starting point for one independent task
- https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/
Supports
- Kubernetes CronJob as the built-in controller for repeated Jobs
- CronJob schedule, overlap, missed-start, suspension, and history behavior
- Idempotency requirement caused by approximate scheduling and possible duplicate creation