> ## Documentation Index > Fetch the complete documentation index at: https://docs.kguardian.dev/llms.txt > Use this file to discover all available pages before exploring further. # Runtime Inventory Endpoints > Which executables and shared libraries each workload container actually runs ## Overview The controller watches `execve` and shared-library mappings per container and reports the distinct set of files it sees, keyed by workload, container and image digest. A path is stored once per (workload, container, digest), not per event, so the inventory grows with what a workload runs rather than how often it runs it. Processes that were already running when the controller started are backfilled from `/proc` and marked `source: "backfill"`; a later runtime sighting upgrades them to `ebpf`. Joined with an image's SBOM on digest and path, this separates packages an image ships from packages it actually loads. Each path also records where the file lived when it ran (`origin`), which is what drift detection uses to flag binaries the image did not ship. ## Enabling it Off by default until its per-node overhead is measured. Set the chart value `controller.runtimeInventory.mode`: | Mode | What is captured | | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `off` (default) | Nothing; no probe is loaded | | `exec` | Executed files (`tp_btf/sched_process_exec`) | | `full` | Executed files plus files mapped executable, i.e. shared libraries and the dynamic loader (`fentry/security_mmap_file`). Falls back to `exec` on a kernel that cannot attach the fentry probe | Each (container, file) is reported once: the probe deduplicates in the kernel, so after warm-up a container costs one hash lookup per exec or executable mapping. Containers already running when the probe attached are backfilled once from `/proc`. Namespaces in `controller.excludedNamespaces` and pods annotated `kguardian.dev/runtime-inventory: "off"` are not recorded. An empty inventory means nothing was observed, not that nothing ran: the feature may be off on that node, the pod may be excluded, or the process may have run before the controller started and exited before the backfill. ## Origin | `origin` | Meaning | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `image` | In an image layer (overlayfs lower layer, never modified) | | `otherFs` | Not on overlayfs: a volume (emptyDir, configMap, hostPath, PVC), or a root filesystem from a non-overlay snapshotter | | `writableLayer` | In the container's writable (overlayfs upper) layer: written or modified after the container started | | `memfd` | An anonymous memory file (`memfd_create`), never on disk | | `deleted` | Unlinked while running | | `unknown` | Not determined: a `/proc` backfill (which cannot see layers; the broker stores every backfill row as `unknown` unless its path shows `deleted` or `memfd`), or overlayfs without its type in kernel BTF | When sightings of one path disagree, the most suspicious wins (in the order `unknown` \< `image` \< `otherFs` \< `deleted` \< `writableLayer` \< `memfd`) and is never downgraded: one replica running the file from its writable layer is the finding, whatever the others did. `deleted`, `writableLayer` and `memfd` together are the "unshipped" origins. A file changed in place while the container runs (an image binary overwritten, which copies it up into the writable layer, or a file unlinked and then executed through an open descriptor) is reported again with its new origin: the probe's dedup key includes the file's state. Not observed: a mapping made executable later with `mprotect(PROT_EXEC)` (JIT compilers, some custom loaders), and code an interpreter reads as data (scripts, `.pyc`, jars). Only `execve` and executable `mmap` are seen. Rows not seen for `RUNTIME_INVENTORY_RETENTION_DAYS` (chart: `broker.runtimeInventory.retentionDays`, default 30) are pruned. ## GET /workloads/\{namespace}/\{kind}/\{name}/runtime Requires the `read` scope when broker auth is on. Returns `200` with an empty `containers` list when nothing has been reported for the workload. | Query | Description | Default | | -------------- | --------------------------------------------------------------------------------------------- | ------- | | `container` | Only this container | all | | `kind` | `exec` or `lib`; anything else is `400` | both | | `origin` | One origin, or `unshipped` for `deleted`, `writableLayer` and `memfd`; anything else is `400` | all | | `limit` | Rows (paths) returned, capped at 5000 | `1000` | | `window_hours` | Coverage window, clamped to 1-720 | `24` | ```json theme={null} { "namespace": "prod", "kind": "Deployment", "name": "web", "containers": [ { "containerName": "app", "imageDigest": "sha256:0123…cdef", "entries": [ { "path": "/usr/sbin/nginx", "kind": "exec", "source": "ebpf", "origin": "image", "pathComplete": true, "firstSeen": "2026-09-26T10:00:00", "lastSeen": "2026-09-27T08:15:00" } ] } ], "truncated": false, "coverage": [ { "containerName": "app", "imageDigest": "sha256:0123…cdef", "instances": 3, "lastHeartbeat": "2026-09-27T08:15:00", "mode": "full", "librariesTracked": true, "covered": true, "observedSince": "2026-09-25T10:00:00", "reason": null, "windowHours": 24 } ] } ``` `coverage` lists every (container, digest) of the workload with a controller heartbeat in the window, with the answer of `kg_runtime_coverage` (see [Coverage](#coverage)). An empty `coverage` means no controller is watching the workload, so an empty inventory says nothing about what runs. One `containers` item per (container, digest): a workload mid-rollout lists the same container once per image. `imageDigest` is `""` when the controller could not resolve the digest. Entries are ordered by `kind`, then `path`. `pathComplete: false` means the kernel-side path walk was cut short and `path` is a suffix. `truncated` is `true` when more rows exist than `limit`. ## GET /images/\{digest}/runtime Requires the `read` scope. The digest must be `sha256:<64 hex>` or `sha512:<128 hex>` (`400` otherwise). Returns every path seen running from the image, aggregated across all workloads that run it. | Query | Description | Default | | -------------- | ------------------------------------------------------------------- | ------- | | `kind` | `exec` or `lib` | both | | `limit` | Rows returned, capped at 5000 | `1000` | | `after` | Resume after this `:`: the previous page's `nextCursor` | start | | `window_hours` | Coverage window, clamped to 1-720 | `24` | ```json theme={null} { "digest": "sha256:0123…cdef", "entries": [ { "path": "/lib/x86_64-linux-gnu/libc.so.6", "kind": "lib", "pathComplete": true, "origin": "image", "firstSeen": "2026-09-20T10:00:00", "lastSeen": "2026-09-27T08:15:00", "workloads": 3 } ], "truncated": false, "nextCursor": null, "coverage": { "workloads": 3, "instances": 7, "lastHeartbeat": "2026-09-27T08:15:00", "librariesTracked": true, "instancesWithDrops": 0, "windowHours": 24 } } ``` When `truncated` is true, `nextCursor` is the `after` value for the next page. `coverage` counts the heartbeating containers running the image in the window; all zero means nobody is watching. `workloads` counts distinct workloads that ran the path. `firstSeen` and `lastSeen` span every workload, and `origin` is the most suspicious one seen in any of them. ## POST /runtime/executables Controller ingest; requires the `ingest` scope. The body is a JSON array of at most 5000 entries (larger batches are refused with `413` and nothing is written; the body limit is 8 MiB). ```json theme={null} [ { "pod_namespace": "prod", "pod_name": "web-7d9f-abcde", "workload_kind": "Deployment", "workload_name": "web", "container_name": "app", "image_digest": "sha256:0123…cdef", "kind": "exec", "path": "/usr/sbin/nginx", "path_complete": true, "source": "ebpf", "origin": "image", "first_seen": "2026-09-26T10:00:00", "last_seen": "2026-09-26T10:05:00" } ] ``` Entries are validated one by one: an entry with an unknown `kind` or `source`, an empty or over-4096-byte path, an invalid digest, or a missing namespace, container or workload is dropped and counted, and the rest of the batch is stored. An entry with no workload is keyed to its pod (`kind: "Pod"`). `origin` is optional: absent, or a value this broker does not know, is stored as `unknown`. Timestamps are UTC; a `last_seen` in the future is clamped to the broker's database clock. ```json theme={null} { "accepted": 1, "dropped": 0, "written": 1 } ``` `written` counts rows inserted or changed. Re-reporting a known path only rewrites its row when `last_seen` moved by five minutes or more, or when the report adds something (an earlier `first_seen`, an eBPF sighting of a backfilled path, a complete path, a more suspicious `origin`), so a steady-state batch writes nothing. ## Coverage An inventory row proves a file ran. It cannot prove that a file did not run, because a node with the feature off writes nothing. For that, each controller heartbeats every container it tracks every 5 minutes, whether or not anything new ran, and the broker answers "was this watched continuously?" with the SQL function ```sql theme={null} kg_runtime_coverage(cluster text, namespace text, kind text, name text, container text, image_digest text, window_hours integer) RETURNS TABLE (covered boolean, observed_since timestamp, reason text) ``` `observed_since` is naive UTC, like every broker timestamp. `covered` is a negative claim ("this never ran") that can end up in a VEX statement, so every doubt answers false. It is true only when all of these hold for the workload container and image: * at least one instance heartbeated within the window; * every such instance had the exec probe, and the library probe in mode `full` (`exec` mode cannot vouch for a library never being loaded); * no instance lost events in the window: the kernel could not queue them, the container hit its 4096-path cap, the broker dropped entries at ingest, or sightings arrived before the pod was known; * no instance still had entries waiting to reach the broker at its last heartbeat; * no path of the container and image is incomplete (a truncated path cannot be matched to the package that owns it), and no instance reported its inventory incomplete: the `/proc` backfill reads at most 4 MiB of a process's `maps`, and a container cut short there stays uncovered for as long as it runs; * every such instance is still heartbeating on time (within 3 × interval + 60 s), or ended cleanly; * every such instance was either captured from its start with no gap since, or covered without a gap since before the window began. A container already running when the probe attached counts from its `/proc` backfill; * no live pod of the workload runs the container without a fresh heartbeat, for example on a node with the feature off; * coverage began at least `window_hours` ago. Otherwise `covered` is false, and `reason` is the first failing check in this order: `no_runtime_data`, `probes_missing`, `libraries_not_tracked`, `events_dropped`, `events_pending`, `incomplete_paths`, `capture_gap`. A late heartbeat or a probe or mode change restarts the gap-free run at the heartbeat that ended it. A lost event keeps every window containing it uncovered. A full dedup map in the kernel loses nothing: it evicts its oldest entry, and that file is reported again on its next use. ## POST /runtime/coverage Controller heartbeats; requires the `ingest` scope. At most 5000 per post, counted while parsing (`413` above that), 4 MiB body. Invalid entries are dropped and counted. A heartbeat older than the stored one is ignored, and a container's row is frozen once a heartbeat with `ended: true` arrives. ```json theme={null} [ { "pod_namespace": "prod", "pod_name": "web-7d9f-abcde", "workload_kind": "Deployment", "workload_name": "web", "container_name": "app", "image_digest": "sha256:0123…cdef", "container_id": "5e1e2d3c…", "node_name": "node-1", "mode": "full", "exec_probe": true, "lib_probe": true, "start_mode": "start", "tracking_since": "2026-09-26T10:00:00", "events_dropped": 0, "unsent": 0, "incomplete": false, "ended": false, "heartbeat_at": "2026-09-27T08:15:00", "heartbeat_secs": 300 } ] ``` Rows not heartbeated for `RUNTIME_INVENTORY_RETENTION_DAYS` are pruned.