eBPF Observability
eBPF observability runs small, verified programs at Linux kernel and application hook points to measure what a system is doing. It can expose network flows, system calls, latency, and stack samples without adding instrumentation to every application.
itObservability and performance | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Intro
eBPF Observability
eBPF observability measures Linux systems by running event-driven programs at selected kernel or application hook points. A hook fires when an event occurs, such as a system call, function entry, tracepoint, network operation, or periodic CPU sample. The attached eBPF program reads the event context, filters or aggregates data, and passes selected results to a user-space collector.
The important boundary is that eBPF is a telemetry mechanism, not an observability backend. It can collect signals near the source with system-wide context. A complete observability system still needs naming, storage, queries, visualization, alerting, and operational ownership.
The architecture
An eBPF observability path has five parts:
- Hook: A defined event location triggers the program. Common choices include tracepoints, kprobes, uprobes, fentry and fexit trampolines, socket hooks, and performance sampling events.
- Program: A short program reads the available context and performs bounded work. The program may filter events, build a key, update an aggregate, capture a stack identifier, or emit a record.
- Verifier and loader: A user-space loader asks the kernel to load the program. The verifier analyzes control flow, register state, pointer use, memory access, and helper use before the kernel accepts it.
- Maps or event buffers: Maps retain shared state and aggregates. Perf buffers or BPF ring buffers carry event records from kernel space to user space.
- Collector and backend: A user-space process reads results, enriches kernel identifiers with process, container, or Kubernetes metadata, then exports or stores telemetry.
This split keeps the hot-path work close to the event while moving expensive formatting, symbolization, correlation, and storage out of the kernel.
The event and data flow
Consider a collector that measures request latency from network activity:
request reaches a hook
|
v
eBPF program reads event context
|
+--> discard events outside the target
|
+--> update a latency histogram in a map
|
+--> emit selected records through a ring buffer
v
user-space collector reads, enriches, and exports telemetry
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://ebpf.io/what-is-ebpf/
Supports
- Event-driven eBPF programs and hook families
- Verification and JIT execution model
- Maps, helper functions, user-space loaders, and observability use cases
- BPF and eBPF naming convention
- https://docs.ebpf.io/
Supports
- Technical documentation index for concepts, program types, maps, helpers, system-call commands, and libraries
- Scope and progression of the reference path
- https://docs.ebpf.io/linux/concepts/
Supports
- Linux concepts including maps, verifier, concurrency, pinning, resource limits, and trampolines
- Compatibility dimensions beyond source code
- https://docs.ebpf.io/linux/program-type/
Supports
- Tracing program-type families and libbpf section conventions
- Program type and attachment as distinct parts of the execution model
- https://docs.ebpf.io/linux/program-type/BPF_PROG_TYPE_TRACEPOINT/
Supports
- Tracepoints as predefined kernel events
- Tracepoint context and attachment model
- https://docs.ebpf.io/linux/program-type/BPF_PROG_TYPE_TRACING/
Supports
- Fentry and fexit trampoline attachment
- Typed entry arguments and exit return values
- Relationship to kprobes and kretprobes
- https://docs.kernel.org/bpf/
Supports
- Primary Linux BPF subsystem documentation path
- Verifier, maps, libbpf, security, testing, and standardization references
- https://docs.kernel.org/bpf/verifier.html
Supports
- Control-flow and execution-path analysis
- Register, pointer, stack, and map-value state tracking
- Verifier acceptance as a kernel safety check
- https://docs.kernel.org/bpf/maps.html
Supports
- Maps as shared storage between BPF programs and user space
- Map creation, lookup, update, deletion, and lifetime
- Finite typed map families used by observability programs
- https://docs.kernel.org/bpf/ringbuf.html
Supports
- Shared cross-CPU event buffer and ordering motivation
- Reservation, commit, discard, and output APIs
- Non-blocking reservation failure and record-loss behavior
- Ring size as a fixed power-of-two capacity
- https://docs.kernel.org/6.10/bpf/libbpf/libbpf_overview.html
Supports
- BTF, compiler, and libbpf roles in BPF CO-RE
- Target-kernel relocation of types, fields, and offsets
- Kernel BTF exposure through the sysfs vmlinux object
- https://docs.ebpf.io/concepts/core/
Supports
- CO-RE definition and portability problem
- BTF and relocation responsibilities
- https://bpftrace.org/docs/release_026/language
Supports
- Tracepoint, kprobe, fentry, fexit, uprobe, USDT, and sampling provider roles
- Fentry and fexit BTF requirements and typed arguments
- Kprobe casting and function-coupling considerations
- https://bpftrace.org/docs/release_026/cli
Supports
- Probe discovery and verbose argument inspection
- Availability checks against the current host
- https://bpftrace.org/docs/release_025/language
Supports
- Scratch variables and BPF-backed maps
- Per-thread entry-to-return correlation pattern
- Bounded map declarations and LRU eviction behavior
- https://docs.kernel.org/bpf/bpf_design_QA.html
Supports
- Kernel internals not forming a stable ABI by default
- Tracing privilege and helper boundaries
- Portability limits for kernel functions and data structures
- https://kernelnewbies.org/Linux_5.8
Supports
- CAP_BPF and CAP_PERFMON capability introduction
- Capability combinations for tracing and networking programs
- BPF ring-buffer milestone in Linux 5.8
- https://opentelemetry.io/docs/zero-code/obi/
Supports
- eBPF application auto-instrumentation for HTTP and gRPC signals
- Zero-code baseline observability and compatibility limits
- Coexistence with application and language instrumentation
- https://grafana.com/docs/beyla/latest/
Supports
- Executable and network inspection for baseline RED metrics and traces
- OpenTelemetry and Prometheus export
- BTF, kernel, capability, language, and tracing limitations
- Application SDKs as the source of deeper spans and custom telemetry
- https://github.com/sindresorhus/awesome
Supports
- Discovery of the curated Awesome eBPF list
- https://github.com/qmonnet/awesome-ebpf
Supports
- Curated discovery of BCC, bpftrace, Inspektor Gadget, Pixie, Parca, Hubble, Caretta, DeepFlow, Coroot, and Kyanos
- Separation of reference documentation, workflow tools, and observability projects
- https://github.com/iovisor/bcc
Supports
- BCC framework, language front ends, examples, and ready-to-run tracing tools
- https://inspektor-gadget.io/docs/main/
Supports
- OCI-packaged eBPF gadgets for Linux and Kubernetes
- Container and Kubernetes enrichment of kernel data
- Collection, export, filtering, and extension roles
- https://docs.px.dev/about-pixie/what-is-pixie/
Supports
- Per-node eBPF collection and in-cluster data plane
- Protocol traces, resource metrics, network metrics, and application profiles
- Kubernetes-focused service and request visibility
- https://docs.px.dev/about-pixie/pixie-ebpf/
Supports
- Kernel probes for network system calls
- TLS-library uprobes for plaintext-side protocol visibility
- Sampling-based application CPU profiling
- https://docs.cilium.io/en/stable/observability/hubble/
Supports
- Node, cluster, and multi-cluster network observability
- Hubble command line, relay, user interface, metrics, and service map
- https://www.parca.dev/docs/overview/
Supports
- eBPF whole-system continuous profiling agent
- Profile storage, label-based queries, comparisons, and incident analysis
- https://github.com/groundcover-com/caretta
Supports
- eBPF-derived Kubernetes service dependency map
- Prometheus-compatible metrics and Grafana visualization
- https://deepflow.io/docs/
Supports
- eBPF service maps, distributed tracing, and continuous profiling
- Correlation of network, request, and stack evidence
- https://coroot.com/docs
Supports
- eBPF application and infrastructure observability scope
- Service maps, request signals, profiling, and operational workflows
- https://coroot.com/pricing
Supports
- Community and commercial editions
- Free community use and paid product tiers
- https://kyanos.io/
Supports
- eBPF network-request capture and troubleshooting
- HTTP, Redis, MySQL, Kafka, MongoDB, and DNS protocol analysis scope
- https://ebpf.io/applications/
Supports
- Community landscape of actively maintained open-source eBPF applications
- Cilium, Pixie, Pyroscope, and Coroot observability roles
- https://cilium.io/use-cases/network-observability/
Supports
- Hubble network-flow, service-dependency, and policy-verdict evidence
- Cilium-managed Kubernetes network scope
- https://docs.groundcover.com/
Supports
- Node-level eBPF sensor and automatic collection
- Customer-environment data plane and unified telemetry path
- https://www.groundcover.com/pricing
Supports
- Free entry option and paid per-node product tiers
- eBPF sensor as the monitored-node unit
- https://docs.datadoghq.com/universal_service_monitoring/
Supports
- eBPF-based service discovery, request metrics, and dependency visibility
- Baseline service monitoring without application code changes
- https://www.datadoghq.com/pricing/
Supports
- Universal Service Monitoring as a commercial Datadog product
- Paid platform packaging and trial path
- https://www.elastic.co/observability/universal-profiling
Supports
- eBPF whole-system continuous profiling
- Application, library, and kernel stack visibility
- Elastic Cloud trial and commercial backend relationship
- https://www.polarsignals.com/
Supports
- Cross-language eBPF continuous profiling
- Flame-graph queries, comparisons, and production profile history
- Commercial cloud trial and product path
- https://www.usenix.org/conference/usenix-winter-1993-conference/presentation/bsd-packet-filter-new-architecture-user-level-
Supports
- January 1993 publication of the BSD Packet Filter architecture
- Register-based filtering and reduced kernel-to-user copying
- https://www.kernel.org/doc/html/v6.0/bpf/bpf_licensing.html
Supports
- Significant extension of the classic BPF instruction set in 2014
- Instruction-set continuity and licensing context
- https://kernelnewbies.org/Linux_3.18
Supports
- December 2014 Linux 3.18 release date
- bpf system call, program loading, maps, helper calls, and JIT milestone
- https://lwn.net/Articles/742082/
Supports
- April 2015 BCC release
- Higher-level front ends and collection of tracing tools
- https://kernelnewbies.org/Linux_4.8
Supports
- October 2016 Linux 4.8 release date
- XDP as a programmable high-performance network data path
- https://kernelnewbies.org/Linux_4.18
Supports
- August 2018 Linux 4.18 release date
- Introduction and initial purpose of BPF Type Format
- https://facebookmicrosites.github.io/bpf/blog/
Supports
- February 2020 public BPF CO-RE portability documentation
- BTF, libbpf, and target-side compilation problem
