Improve checkout performance on Windows runners by upgrading @actions/github dependency by BrettDong · Pull Request #1246 · actions/checkout

BrettDong · 2023-03-26T11:30:06Z

@dscho discovered that the checkout action could stall for a considerable amount of time on Windows runners waiting for PowerShell invocations made from 'windows-release' npm package to complete.

Then I studied the dependency chain to figure out where 'windows-release' was imported:

'@actions/checkout'
  <- '@actions/github'@2.2.0
    <- '@octokit/endpoint'@6.0.1
    <- '@octokit/graphql'@4.3.1
    <- '@octokit/request'@5.4.2
    <- '@octokit/rest'@16.43.1
      <- 'universal-user-agent'@4.0.1
        <- 'os-name'@3.1.0
          <- 'windows-release'@3.1.0

'universal-user-agent' package dropped its dependency on 'os-name' in v6 (https://github.com/gr2m/universal-user-agent/releases/tag/v6.0.0).

'@actions/github' v3 removed dependency on '@octokit/rest'@16.43.1 and allows users to move away from the old 'universal-user-agent' v4. (actions/toolkit#453)

This pull request attempts to update the version of '@actions/github' used in the checkout action to avoid importing 'windows-release'.

Based on testing in my own repositories, I can see an improvement in reduced wait time between entering the checkout action and git actually starts to do useful work.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

dscho

I confirm that this fixes the abysmal slowness on Windows that was reported in #1186 and that I was able to reproduce.

Well done, @BrettDong!

briantist · 2023-03-26T20:20:36Z

Eagerly awaiting a fix for this; for those of us paying for runners it's not just a convenience, we end up paying significantly more because the checkout step takes the majority of the runner's time, and with a workflow that has a lot of parallel runs, this adds up really quickly.

briantist · 2023-03-29T21:16:28Z

I've been using this and it was faster, but I just had a run with it that took 1m41s 😖

BrettDong · 2023-04-04T07:29:19Z

Hi @thboop , could you please re-approve the CI checks? Thanks!

fhammerl · 2023-04-06T13:59:52Z

Hi @BrettDong, thanks for this PR! Updating to actions/github@v5 is both something we want to do and it sidesteps the whole issue by eliminating our dependency on windows-release by removing universal-user-agent all together, so thanks for taking that approach :)

A few notes / questions:

Do you have a minimal repro that only requires one of windows-release? E.g. I tried to run console.log(windowsRelease()) on a larger runner but it did not cause the same stalling behaviour. Running (Get-CimInstance -ClassName Win32_OperatingSystem).caption directly in powershell is also instantaneous
I could only reproduce this issue on larger runners, not on windows-latest or windows-2019. Do you experience it there too?

BrettDong · 2023-04-06T15:46:43Z

The delay from windows-release is only part of the reason checkout is slow on Windows. This pull request solves specifically this single issue. In majority of cases the delay of windows-release is of single digit seconds; in rarer cases it takes longer time. I don't know why the time is indeterministic. But at least this pull request completely eliminates the PowerShell invocation and saves at least a couple of seconds in each checkout run.

Apart from the windows-release problem discovered by @dscho, another source of stall I discovered is extremely slow disk I/O for loading NodeJS and Git executable and libraries: #1186 (comment). But I don't think it's something can be improved by external users like us.

@dscho

…/github dependency Re: #1186 @dscho discovered that the checkout action could stall for a considerable amount of time on Windows runners waiting for PowerShell invocations made from 'windows-release' npm package to complete. Then I studied the dependency chain to figure out where 'windows-release' was imported: '@actions/checkout'@main <- '@actions/github'@2.2.0 <- '@octokit/endpoint'@6.0.1 <- '@octokit/graphql'@4.3.1 <- '@octokit/request'@5.4.2 <- '@octokit/rest'@16.43.1 <- 'universal-user-agent'@4.0.1 <- 'os-name'@3.1.0 <- 'windows-release'@3.1.0 'universal-user-agent' package dropped its dependency on 'os-name' in https://github.com/gr2m/universal-user-agent/releases/tag/v6.0.0 . '@actions/github' v3 removed dependency on '@octokit/rest'@16.43.1 and allows users to move away from the old 'universal-user-agent' v4. (actions/toolkit#453) This pull request attempts to update the version of '@actions/github' used in the checkout action to avoid importing 'windows-release'. Based on testing in my own repositories, I can see an improvement in reduced wait time between entering the checkout action and git actually starts to do useful work.

BrettDong · 2023-04-06T15:56:24Z

I could only reproduce this issue on larger runners, not on windows-latest or windows-2019. Do you experience it there too?

I never used large runners and I don't understand what issue you are referring to.

thboop · 2023-04-10T17:49:14Z

Hey @BrettDong , thanks for putting this together, I'd love to get this merged and released.

It looks like the breaking change in the new major version of actions/github is causing this new behavior to fail. Do you have some time to update this pr, or should we take it over?

BrettDong · 2023-04-11T12:13:35Z

I did not run npm build last time and it might be the cause of test failures. @thboop: Could you approve CI checks?

fhammerl

LGTM!

fhammerl · 2023-04-12T11:57:25Z

Thanks for your contribution @BrettDong

Small side-note, did you need to manually recognise the MIT license of .licenses/npm/@actions/http-client-1.0.8.dep.yml or did licensed work the first time?

BrettDong · 2023-04-12T12:29:19Z

Thanks for your contribution @BrettDong

Small side-note, did you need to manually recognise the MIT license of .licenses/npm/@actions/http-client-1.0.8.dep.yml or did licensed work the first time?

I manually updated the yaml, following licensed documentation.

The PR actions#1246 replaced the `getOctokit` method from the `octokit-provider.ts` file with the `getOctokit` method from the `@actions/github` package. The octokit-provider was previously responsible for creating an Octokit instance and setting the `baseUrl` via the `getServerApiUrl` helper function. This function calls `getServerUrl` which reads the server url from the `GITHUB_SERVER_URL` environment variable, which on GHES is set to the enterprise instance. This commit restores the previous behaviour by calling `getServerApiUrl` in all places where an octokit instance is created. Co-authored-by: Markus Wolf <mail@markus-wolf.de>

ZauberNerd · 2023-04-13T12:16:48Z

This PR breaks GHES compatibility in case actions/checkout is used in an environment where no git binary exists. See also: #1288

BrettDong requested a review from a team as a code owner March 26, 2023 11:30

BrettDong mentioned this pull request Mar 26, 2023

Checkout suddenly much slower on windows environment #1186

Open

10 tasks

dscho added a commit to dscho/git-for-windows-automation that referenced this pull request Mar 26, 2023

DEBUG: test actions/checkout#1246

24f8ba5

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

dscho approved these changes Mar 26, 2023

View reviewed changes

dscho reviewed Mar 31, 2023

View reviewed changes

Comment thread src/github-api-helper.ts Outdated

Spiritaine approved these changes Apr 5, 2023

View reviewed changes

fhammerl self-assigned this Apr 6, 2023

BrettDong commented Apr 6, 2023

View reviewed changes

Comment thread src/github-api-helper.ts Outdated

BrettDong added 2 commits April 6, 2023 23:52

Update .licenses

7ac4fa6

Rebuild index.js

f6d3681

fhammerl reviewed Apr 12, 2023

View reviewed changes

Comment thread src/github-api-helper.ts

fhammerl approved these changes Apr 12, 2023

View reviewed changes

fhammerl merged commit 40a16eb into actions:main Apr 12, 2023

1newsr reviewed Apr 13, 2023

View reviewed changes

Comment thread src/github-api-helper.ts

ZauberNerd mentioned this pull request Apr 13, 2023

Pass correct baseUrl to octokit #1288

Closed

fhammerl mentioned this pull request Apr 13, 2023

Fix: convert baseUrl to ser

Navigation Menu

Uh oh!

Improve checkout performance on Windows runners by upgrading @actions/github dependency - #1246

Improve checkout performance on Windows runners by upgrading @actions/github dependency#1246
fhammerl merged 3 commits into
actions:mainfrom
BrettDong:octokit

BrettDong commented Mar 26, 2023

Uh oh!

dscho left a comment

Uh oh!

briantist commented Mar 26, 2023

Uh oh!

briantist commented Mar 29, 2023

Uh oh!

Uh oh!

BrettDong commented Apr 4, 2023

Uh oh!

Uh oh!

fhammerl commented Apr 6, 2023

Uh oh!

BrettDong commented Apr 6, 2023

Uh oh!

BrettDong commented Apr 6, 2023

Uh oh!

thboop commented Apr 10, 2023 •

edited

Loading

Uh oh!

BrettDong commented Apr 11, 2023

Uh oh!

Uh oh!

fhammerl left a comment

Uh oh!

fhammerl commented Apr 12, 2023

Uh oh!

BrettDong commented Apr 12, 2023

Uh oh!

Uh oh!

ZauberNerd commented Apr 13, 2023

Uh oh!

Uh oh!

Conversation

BrettDong commented Mar 26, 2023

Uh oh!

dscho left a comment

Choose a reason for hiding this comment

Uh oh!

briantist commented Mar 26, 2023

Uh oh!

briantist commented Mar 29, 2023

Uh oh!

Uh oh!

BrettDong commented Apr 4, 2023

Uh oh!

Uh oh!

fhammerl commented Apr 6, 2023

Uh oh!

BrettDong commented Apr 6, 2023

Uh oh!

BrettDong commented Apr 6, 2023

Uh oh!

thboop commented Apr 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BrettDong commented Apr 11, 2023

Uh oh!

Uh oh!

fhammerl left a comment

Choose a reason for hiding this comment

Uh oh!

fhammerl commented Apr 12, 2023

Uh oh!

BrettDong commented Apr 12, 2023

Uh oh!

Uh oh!

ZauberNerd commented Apr 13, 2023

Uh oh!

thboop commented Apr 10, 2023 •

edited

Loading