Skip to content

Fix orphaned processes on Windows when quickly navigating between commits - #5885

Merged
stefanhaller merged 4 commits into
masterfrom
fix-windows-pty-orphans
Aug 4, 2026
Merged

Fix orphaned processes on Windows when quickly navigating between commits#5885
stefanhaller merged 4 commits into
masterfrom
fix-windows-pty-orphans

Conversation

@stefanhaller

Copy link
Copy Markdown
Collaborator

Fixes #5879: with an external diff command configured, quickly navigating between commits on Windows accumulates orphaned git.exe/difft.exe/conhost.exe processes that keep computing their diffs in the background and persist after lazygit exits.

Stopping a pty task on Windows relied on ClosePseudoConsole, whose CTRL_CLOSE_EVENT only reaches clients attached to the pseudoconsole at that moment. Attachment happens asynchronously during child startup, so a task stopped within the first few milliseconds of its life — which is exactly what rapid navigation produces — misses the event entirely and survives, together with its whole process tree (git for Windows spawns through a two-level git.exe wrapper, so a single task has several attach windows).

The fix puts the child into a job object before it executes its first instruction (created suspended → assigned → resumed), so every descendant is in the job from the start. The pty teardown still closes the pseudoconsole first and gives clients that received the close event a moment (500ms) to exit through their own handlers — git's cleans up lock files — and then terminates whatever is left in the job. JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE doubles as a safety net: if lazygit exits without running the teardown, the OS closes the job handle and reaps the tree.

Validated with a harness that mimics lazygit's stop path with randomized 0–120ms stop delays: before the fix, 3 of 30 process trees survived as permanent orphans per run; after it, three runs of 30 all came back with zero orphans, with the job kill catching exactly the children that missed the close event (6 of 90) while the rest still exited gracefully through the event as before.

@stefanhaller stefanhaller added the bug Something isn't working label Aug 2, 2026
@stefanhaller stefanhaller mentioned this pull request Aug 2, 2026
@stefanhaller
stefanhaller force-pushed the fix-windows-pty-orphans branch 3 times, most recently from 323b0e5 to 2528e42 Compare August 3, 2026 17:13
@stefanhaller
stefanhaller changed the base branch from master to prevent-stale-index-lock-windows August 3, 2026 17:15
@stefanhaller
stefanhaller force-pushed the fix-windows-pty-orphans branch from 2528e42 to bc56716 Compare August 3, 2026 17:31
@stefanhaller
stefanhaller force-pushed the fix-windows-pty-orphans branch from bc56716 to 5b0ecf6 Compare August 4, 2026 04:54
Base automatically changed from prevent-stale-index-lock-windows to master August 4, 2026 05:03
stefanhaller and others added 4 commits August 4, 2026 07:03
The task stop path terminates the still-running command by pulling its
*os.Process out of the Cmd interface and applying one global strategy
(TerminateProcessGracefully) to it. That shape can't accommodate the
upcoming fix for orphaned process trees on Windows: there, stopping a
pty task requires terminating the entire process tree via a job object
whose handle lives with the pty, not with the process. And the two Cmd
implementations genuinely need different strategies anyway: a
process-group kill (the likely future fix for #5675 on Unix) is only
safe for pty children, which run as session leaders, while plain
commands share lazygit's own process group.

So let each Cmd implementation decide how to terminate itself, and drop
GetProcess, which had no other callers. No change in behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stopping a pty task on Windows relies on ClosePseudoConsole, which
delivers CTRL_CLOSE_EVENT to the console's attached clients. But only
to those attached at that moment: when the user flicks quickly through
commits, a task is often stopped within the first few milliseconds of
its life, before the child has attached to the pseudoconsole. Such a
child misses the event and survives, running the entire diff to
completion in the background (spawning one external differ per changed
file) and keeping its conhost.exe alive; rapid navigation accumulates
these git/difft/conhost trees, and they outlive lazygit. Grandchildren
are affected too: git for Windows runs commands through a two-level
git.exe wrapper, so a single task has several attach windows, and a
grandchild spawned while the console is going down is orphaned even
when its parent got the event and exited.

Fix this by putting the child into a job object before it runs its
first instruction (created suspended, assigned, then resumed), so that
every descendant is in the job from the start; the teardown in Close
terminates the job right after initiating the pseudoconsole close.
There is no point in a grace period between the two: the close event
is not a graceful signal -- git and the common diff tools leave it to
the default handler, which calls ExitProcess at an arbitrary point --
so clients that received it are already dying, and the kill exists for
those that missed it. Killing at an arbitrary point cannot leak a
stale index.lock, because pty-rendered commands no longer take that
lock (see withPtyGitConfig in pkg/gui/pty.go).

The pseudoconsole close runs on its own goroutine because the kill
must not wait for it: on builds where ClosePseudoConsole blocks until
the console host exits (pre-24H2), the host keeps running as long as a
surviving client does, and that client only goes away through the job
kill; sequencing the kill after a blocking close would deadlock in
exactly the case the kill exists for.

KILL_ON_JOB_CLOSE doubles as a safety net: if lazygit exits without
running the teardown, the OS closes the job handle and reaps the tree.

In a harness that mimicked the stop path with randomized 0-120ms stop
delays, 3 of 30 process trees survived as orphans before this change;
none survive with it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…5879)

A pseudoconsole's conhost.exe is spawned by CreatePseudoConsole as a
child of lazygit, so it is not part of the job object that the pty
teardown kills. That is normally fine: a healthy conhost runs itself
down once the reference handle is closed and its clients are gone. But
conhost builds before the ConPTY overhaul that shipped with Windows 11
24H2 (confirmed on 23H2, build 22631) fail to complete the rundown
when a client attached after the close event was delivered and was
then killed -- the fate of exactly the clients the job kill exists for
-- and such a conhost lingers forever with no clients, at a rate of
about one per five fast commit navigations. These builds remain
widespread: all of Windows 10 (whose ESU tail runs into 2028, and
whose hardware often cannot run Windows 11 at all) plus pre-24H2
Windows 11 fleets.

Since Windows offers no way to obtain the conhost's pid or handle from
the HPCON, identify it by diffing lazygit's direct conhost children
around the CreatePseudoConsole call, serialized by a mutex so that two
concurrently starting ptys can't confuse each other's diff, and open a
handle immediately so that pid reuse is harmless. The teardown then
gives conhost a second to exit on its own before terminating it; on
healthy builds the wait succeeds and the reap never fires. If the
conhost can't be identified unambiguously, we simply don't reap, which
is no worse than before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pty teardown in Close runs on a background goroutine that doesn't
get to finish when lazygit is quitting: the process exits milliseconds
after the view buffer managers are closed. The job objects still cover
the clients -- KILL_ON_JOB_CLOSE reaps them when the process's handles
are rundown at exit -- but nothing reaps the conhost, so on Windows
builds whose conhost fails to run down on its own, quitting leaks one
conhost per live pty.

This is not a rare timing window: a diff longer than what has been
read keeps its git process (and thus its pty and conhost) running for
the entire time it is displayed, so that scrolling can read more.
Quitting while looking at a long diff is therefore the common case,
and with an external differ configured it leaks a conhost on affected
builds on almost every quit.

Fix this by having the gui's shutdown path wait synchronously for the
in-flight teardowns after closing the view buffer managers. A quit
signal makes the teardowns skip the conhost rundown wait -- the
conhost serves nothing once its clients are dead, and the exit must
not stall for its sake -- so the wait normally completes in
milliseconds, keeping quit as fast as before; a 2-second cap protects
the exit path even if a teardown wedges.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@stefanhaller
stefanhaller force-pushed the fix-windows-pty-orphans branch from 5b0ecf6 to d251fad Compare August 4, 2026 05:03
@stefanhaller
stefanhaller merged commit ccdfa8f into master Aug 4, 2026
12 of 13 checks passed
@stefanhaller
stefanhaller deleted the fix-windows-pty-orphans branch August 4, 2026 05:06
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Aug 11, 2026
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [jesseduffield/lazygit](https://github.com/jesseduffield/lazygit) | minor | `v0.63.1` → `v0.64.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>jesseduffield/lazygit (jesseduffield/lazygit)</summary>

### [`v0.64.0`](https://github.com/jesseduffield/lazygit/releases/tag/v0.64.0)

[Compare Source](jesseduffield/lazygit@v0.63.1...v0.64.0)

<!-- Release notes generated using configuration in .github/release.yml at v0.64.0 -->

#### What's Changed

This release has massive changes, but most of them should hopefully not be visible: I completely overhauled lazygit's concurrency model, which was, let's say, less than robust; there were lots of data races, and we were just lucky that this didn't result in crashes or misbehavior more often. We now have a robust concurrency model with no known data races, and in fact we run our integration test suite on CI with the `-race` flag to prove that. The user visible part of this is that some operations run a little more smoothly now; for example, there used to be an ugly spinner freeze at the end of checking out a branch, which is now gone.

However, since the changes were so massive there's a higher-than-usual chance of regressions, so please report any that you find.

Apart from that, we also have a few useful enhancements; the most notable one is probably that we now show the Github checks status of pull requests in the branches panel.

##### Enhancements 🔥

- Show a spinner for more long-running operations by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5765](jesseduffield/lazygit#5765)
- Remove the BLOCK\_UI refresh mode by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5790](jesseduffield/lazygit#5790)
- Support a `{{diffContext}}` template variable in external diff command by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5841](jesseduffield/lazygit#5841)
- Some small UI polish by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5853](jesseduffield/lazygit#5853)
- Auto-scroll when dragging to create range selection in staging view by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5855](jesseduffield/lazygit#5855)
- Create a range selection in list views by dragging with the mouse by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5856](jesseduffield/lazygit#5856)
- Reorder commits (or rebase todos) by dragging with the mouse by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5857](jesseduffield/lazygit#5857)
- Rework the custom pager config (rename to diff renderer) by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5870](jesseduffield/lazygit#5870)
- Show a Github MR's combined checks state in branches list (and main view for selected branch) by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5874](jesseduffield/lazygit#5874)

##### Fixes 🔧

- Fix stuck inline status when pushing/fetching by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5768](jesseduffield/lazygit#5768)
- Escape the merge conflicts view before prompting to continue the rebase by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5822](jesseduffield/lazygit#5822)
- Fix side panel rendering when branches/commits are not their panel's first tab by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5825](jesseduffield/lazygit#5825)
- Suppress output from a few git commands that pollute the command log by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5834](jesseduffield/lazygit#5834)
- Fix stall with ctrl+z and fg by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5830](jesseduffield/lazygit#5830)
- Fix more problems related to concurrent repo switch and background refresh by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5839](jesseduffield/lazygit#5839)
- Fix Windows crash when switching to fullscreen mode with a custom pager by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5838](jesseduffield/lazygit#5838)
- Fix multi-selection of files with common prefix not working in commit files panel by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5868](jesseduffield/lazygit#5868)
- Support absolute paths when detecting edit preset from EDITOR env var by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5876](jesseduffield/lazygit#5876)
- Exclude more commit trailers from auto-wrapping by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5871](jesseduffield/lazygit#5871)
- Prevent stale index.lock files from diffs rendered through a pty on Windows by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5888](jesseduffield/lazygit#5888)
- Fix orphaned processes on Windows when quickly navigating between commits by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5885](jesseduffield/lazygit#5885)

##### Maintenance ⚙️

- Perform refresh model and view updates on the UI thread instead of using mutexes by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5767](jesseduffield/lazygit#5767)
- Fix data race with status string by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5777](jesseduffield/lazygit#5777)
- Fix data race with command log by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5779](jesseduffield/lazygit#5779)
- Make integration tests using commits more robust by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5782](jesseduffield/lazygit#5782)
- Synchronize ViewBufferManager.Close with a starting task by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5786](jesseduffield/lazygit#5786)
- Make model<->view index conversions independent of rendering by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5785](jesseduffield/lazygit#5785)
- Fix idle notification deadlock by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5821](jesseduffield/lazygit#5821)
- Bump tcell to an unreleased snapshot to fix a shutdown race by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5824](jesseduffield/lazygit#5824)
- Fix command log streaming race by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5789](jesseduffield/lazygit#5789)
- Synchronize async view rendering by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5791](jesseduffield/lazygit#5791)
- Run tests with race detection on CI by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5792](jesseduffield/lazygit#5792)
- Gocui mouse event fixes by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5854](jesseduffield/lazygit#5854)
- Move Github MR cache out of state.yml into a separate file by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5884](jesseduffield/lazygit#5884)
- Make justfile commands available in the Nix development shell by [@&#8203;TyceHerrman](https://github.com/TyceHerrman) in [#&#8203;5890](jesseduffield/lazygit#5890)

##### Docs 📖

- Clarify contribution policy by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5809](jesseduffield/lazygit#5809)

##### I18n 🌎

- Update translations from Crowdin by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5891](jesseduffield/lazygit#5891)

##### Performance Improvements 📊

- Make scrolling down a very long diff with the scroll wheel much smoother by [@&#8203;stefanhaller](https://github.com/stefanhaller) in [#&#8203;5780](jesseduffield/lazygit#5780)

#### New Contributors

- [@&#8203;TyceHerrman](https://github.com/TyceHerrman) made their first contribution in [#&#8203;5890](jesseduffield/lazygit#5890)

**Full Changelog**: <jesseduffield/lazygit@v0.63.1...v0.64.0>

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yODguMCIsInVwZGF0ZWRJblZlciI6IjQzLjI4OC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6bWlub3IiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orphans on Windows

1 participant