Skip to content

feat(memtrack): collect RSS via rss_stat and folio-rmap reconstruction - #453

Merged
not-matthias merged 6 commits into
mainfrom
cod-3089-collect-rss-in-memtrack
Aug 24, 2026
Merged

feat(memtrack): collect RSS via rss_stat and folio-rmap reconstruction#453
not-matthias merged 6 commits into
mainfrom
cod-3089-collect-rss-in-memtrack

Conversation

@not-matthias

@not-matthias not-matthias commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

Adds RSS (resident set size) collection to memtrack, in two layers:

  1. Authoritative RSS from the kernel's kmem:rss_stat tracepoint — absolute per-mm resident bytes (anon/file/shmem/swap), latest-wins.
  2. Reconstructed anonymous RSS from raw kernel folio-rmap events: fentry hooks on the anon folio-rmap add/remove functions emit signed page-count deltas, so anon RSS can be rebuilt over time as Σ(add − remove) × PAGE_SIZE.

The reconstruction is a total anon RSS delta-sum, not a per-vaddr resident map — the kernel remove hook (folio_remove_rmap_ptes) carries no address, so removals can't be attributed to a vaddr (the add hooks' faulting vaddr is emitted for observability only).

Commits

  • feat(memtrack): track RSS via kmem:rss_stat tracepoint — the baseline: EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, parser arm, writer bench case, gated integration test.
  • feat(memtrack): reconstruct anon RSS from gated folio rmap fentry hooksEVENT_TYPE_RMAP_ANON + RmapAnon event, five fentry programs (add_new / add_ptes / remove_ptes / remove_pmd / remove_pud), CO-RE folio helpers, and the load/attach gating.
  • test(memtrack): validate anon RSS reconstruction against rss_stat — ramps anon RSS via mmap/munmap and asserts the reconstructed estimate tracks the rss_stat MM_ANONPAGES peak within 25%.

What's on by default vs gated

  • rss_stat tracepoint: always on. Emitting Rss events is the intended new default behavior introduced by this change — the RSS tracepoint is not gated.
  • RmapAnon folio-rmap fentry programs: off by default, gated behind CODSPEED_MEMTRACK_TRACK_RMAP=1. When the flag is unset they are set_autoload(false) before load and never attached, so:
    • the skeleton still loads on any kernel (a missing fentry BTF target would otherwise fail the whole load), and
    • the folio-rmap reconstruction path stays out of production (--mode memory) and out of the existing test suites — no RmapAnon events are produced by default.

Verification

Run in a privileged, --pid=host container sharing the host kernel (7.0.12):

  • Reconstruction (flag on):real anon amplitude = 64 MiB, estimated peak = 64 MiB (ratio 1.00).
  • Regression (rss_tests, flag unset): ✅ passes — no RmapAnon events, folio-rmap programs stay unloaded.
  • Parser unit tests, cargo fmt, and clippy clean.
  • Kernel BTF signatures for all five folio_*_rmap* functions verified to match the BPF_PROG arg layouts.

Review notes (draft)

  • track_command ordering: the shared test helper spawns the child before enable()/track(root_pid). In practice the child's fork→execve→ld.so→libc-init far outlasts the two BPF-map updates, so tracking is armed before the workload allocates (both fixtures captured full event streams). Flagging in case we'd prefer a leading settle-usleep in the fixtures or an enable-before-spawn change in the helper.
  • PAGE_SIZE: hardcoded to 4096 (correct on x86_64). On a 16K/64K-page arm64 runner the estimate would need sysconf(_SC_PAGESIZE); rss_stat is already in bytes and unaffected. Happy to switch to sysconf if these tests run on arm64 CI.
  • The chart/inspection tooling used during development lives outside the tree (dev artifact) and is not part of this PR.

@codspeed-hq

codspeed-hq Bot commented Jul 13, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 17 untouched benchmarks


Comparing cod-3089-collect-rss-in-memtrack (f6e60e6) with main (1dcc738)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds RSS collection to memtrack. The main changes are:

  • New RSS, rmap, and lifecycle event types.
  • kmem:rss_stat sampling for absolute resident counters.
  • Opt-in folio-rmap hooks for reconstructed RSS deltas.
  • Parser, artifact, CI, and integration-test coverage for RSS tracking.

Confidence Score: 4/5

This is close, but the optional rmap startup path should be fixed before merging.

  • RSS tracepoint attachment is now handled as optional.
  • The test helper changes avoid the earlier process-wide environment mutation problem.
  • Opt-in rmap can still fail tracker startup on kernels where the PUD targets are not present.

Files Needing Attention: crates/memtrack/src/ebpf/memtrack/rmap.rs and crates/memtrack/src/ebpf/memtrack/tracking.rs

Important Files Changed

Filename Overview
crates/memtrack/src/ebpf/memtrack/rmap.rs Adds rmap support detection based on kernel version, but the PUD tier can be enabled when the actual fentry targets are unavailable.
crates/memtrack/src/ebpf/memtrack/tracking.rs Attaches lifecycle, RSS, and rmap programs; optional PUD rmap attach errors still stop tracker startup.
crates/memtrack/tests/shared.rs Adds RSS/rmap test helpers that pass rmap enablement directly instead of mutating process-wide environment state.

Fix all with Greploop Fix All in Claude Code Fix All in Codex

Prompt To Fix All With AI
### Issue 1
crates/memtrack/src/ebpf/memtrack/rmap.rs:54
**PUD hooks can abort startup**

This treats every 6.15+ kernel as having the PUD rmap fentry targets, but those targets depend on kernel config and architecture support. When `CODSPEED_MEMTRACK_TRACK_RMAP=1` runs on a 6.15+ kernel without the PUD pair, `CoreAndPud` keeps those programs enabled and `attach_tracepoints()` propagates the first missing-target attach error. That stops the tracker instead of falling back to the working PTE/PMD rmap hooks. Please detect the actual PUD BTF targets, or make only the PUD attaches best-effort while keeping core rmap support active.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (29): Last reviewed commit: "ci(memtrack): run bpf tests on arm and a..." | Re-trigger Greptile

Comment thread crates/memtrack/tests/rss_reconstruction_tests.rs Outdated
Comment thread crates/memtrack/tests/rss_reconstruction_tests.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated
Comment thread crates/memtrack/tests/rss_reconstruction_tests.rs Outdated
Comment thread crates/memtrack/tests/shared.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from b670b0a to 2f41984 Compare July 13, 2026 17:32
@not-matthias
not-matthias marked this pull request as ready for review July 14, 2026 15:38
Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from 41945a5 to 64688a9 Compare July 17, 2026 14:23
Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from b02f3cb to 411b713 Compare July 17, 2026 18:12
@not-matthias
not-matthias changed the base branch from main to cod-1801-only-attach-to-used-libraries-in-memtrack July 20, 2026 09:07
Base automatically changed from cod-1801-only-attach-to-used-libraries-in-memtrack to main July 20, 2026 13:12
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from e00fb6b to a68424a Compare July 20, 2026 16:08
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from e48a0a0 to f305358 Compare July 20, 2026 16:28
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
@not-matthias

Copy link
Copy Markdown
Member Author

@greptileai pause reviews for now until i enable them again

@greptile-apps

greptile-apps Bot commented Jul 20, 2026

Copy link
Copy Markdown

Understood — pausing reviews. I'll hold off until you re-enable them.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from f4df079 to 921e469 Compare July 22, 2026 14:24
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from fa0526f to e3a67b4 Compare July 22, 2026 17:09
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from e3a67b4 to 2024266 Compare July 23, 2026 17:39
Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 4 times, most recently from 29fa3da to e5b96d3 Compare July 30, 2026 15:35
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 3 times, most recently from 205b479 to 0b9565e Compare August 19, 2026 14:57
@not-matthias

Copy link
Copy Markdown
Member Author

@greptileai review again, focus on the eBPF code (mostly RMAP tracking), be exhaustive

Comment thread crates/memtrack/src/ebpf/memtrack/tracking.rs Outdated

@GuillaumeLagrange GuillaumeLagrange left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

olgtm, obviously quite a big chunk to chew, but looks mostly fine to me. Main point of concern is the maintainability of "which function can be track for which kernel version" which IMO could be improved.

Comment thread crates/memtrack/src/ebpf/c/rss.bpf.h
Comment thread crates/memtrack/src/ebpf/c/rss.bpf.h Outdated
Comment thread crates/memtrack/src/ebpf/c/rss.bpf.h Outdated
Comment thread crates/memtrack/src/ebpf/c/rss.bpf.h Outdated
Comment thread crates/memtrack/src/ebpf/c/rss.bpf.h Outdated
Comment thread crates/memtrack/src/ebpf/c/utils/process_tracking.h
Comment thread crates/memtrack/src/ebpf/memtrack/maps.rs Outdated
Comment thread crates/memtrack/src/ebpf/c/utils/event_helpers.h Outdated
Comment thread crates/memtrack/src/ebpf/tracker.rs
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from 7573f98 to b5a8e9e Compare August 21, 2026 17:43
Set AllowShortFunctionsOnASingleLine: None in .clang-format and apply it, reformatting the allocator uprobe macros accordingly.
A forked child's inherited RSS is invisible to rss_stat: the fork-time
counter copies fire outside the child's context, and anon COW faults
are counter-neutral, so a child that only touches inherited memory
never reports anything on its own. A fork event carrying the parent
pid lets consumers seed the child from the parent's last absolutes;
exec and exit mark where the address space is replaced or torn down.
Sample the kernel's per-mm resident counter through the kmem:rss_stat tracepoint, emitting absolute byte values per mm member. Adds the EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, the parser arm, and a writer bench case.

An rss_stat update from reclaim or another process's madvise fires in the actor's context; track (mm_id, member) -> owning pid so those updates reach the owner. External events may only lower a counter, so stale reads and mm_id collisions cannot invent peaks.
Attach fentry hooks on the folio-rmap add/remove functions, emitting
signed page-count deltas per MM_* bucket so anon, file, and shmem RSS
can be reconstructed over time. Gated behind
CODSPEED_MEMTRACK_TRACK_RMAP; the programs stay autoload-off by default
so the skeleton loads on any kernel, with the PUD pair (only present
since v6.15) gated separately from the core set so rmap still works on
older kernels. Adds the EVENT_TYPE_RMAP contract, MemtrackEventKind::Rmap,
parser arm, and bench case.

Recover the owning pid for rmap events run by another task (reclaim,
process_madvise, khugepaged, KSM) from the mm_struct pointer, and
maintain the ownership maps across exec and thread-group exit. The same
ownership binding also validates external (curr==0) rss_stat updates,
so a stale mm can no longer attribute a counter to the wrong pid.
Add the rss_tests integration suite: per-workload RSS/rmap reconstruction
snapshots against /proc ground truth, fork-seeded child RSS, exec/exit
resets, foreign-actor rmap attribution (reclaim, external madvise), and
mm-ownership across CLONE_VM and exec. Extend tests/shared.rs with the
tracker/fixture helpers these tests need and move compile_c_source into
it for reuse.

The suite needs two surfaces the production paths don't: a tracker mode
that skips the allocator probes and exec-mapping watcher, and readers for
the mm-ownership maps.
Add an aarch64 lane to the bpf-tests matrix and run the rss integration tests alongside the existing test binaries.
@not-matthias
not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from b5a8e9e to f6e60e6 Compare August 24, 2026 07:57
@not-matthias

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/memtrack/src/ebpf/memtrack/rmap.rs
@not-matthias
not-matthias merged commit 00a5923 into main Aug 24, 2026
85 of 87 checks passed
@not-matthias
not-matthias deleted the cod-3089-collect-rss-in-memtrack branch August 24, 2026 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants