In this commit, we significantly expand the cleanup-space GitHub Actions
workflow to free up substantially more disk space on GitHub runners. The
previous cleanup only removed three large toolsets (dotnet, android,
ghc), which should free ~14GB. This enhancement adds removal of several
additional large packages and caches, bringing the total freed space to
approximately 20-25GB.
The specific additions include removing Swift and Julia language runtimes,
the hosted toolcache directory, all Docker images, numerous large apt
packages (aspnetcore, llvm, php, mongodb, mysql, azure-cli, browsers, and
development tools), and various cache directories. We also add disk space
reporting before and after cleanup to provide visibility into how much
space is actually being freed during workflow runs.
This enhancement was motivated by release builds running out of disk space
when building for all 15 supported platforms (darwin, freebsd, linux,
netbsd, openbsd, windows across multiple architectures). The sequential
builds with verbose output were consuming more space than the basic cleanup
could provide.
In this commit, we revamp the way our skip-labels worked to instead mark
certain stages as green, instead of skipping them. Skipped changes
didn't count towards the set of required checks, which slowed down PR
velocity.
Previously, when PRs were labeled with 'no-itest' or 'no-changelog',
the corresponding CI jobs would be completely skipped. This caused the
GitHub checks to show as skipped rather than successful, which could
be confusing and prevented certain merge rules from working properly.
This commit changes the behavior so that these jobs still run but
immediately report success when the special labels are detected. Each
affected job now starts with a label check step that sets a skip
flag, and all subsequent steps are conditionally executed based on
this flag. When skipped, the jobs add a notice to the GitHub step
summary explaining that tests were auto-passed due to the label.
The change affects five jobs: basic-integration-test, integration-test,
windows-integration-test, macos-integration-test, and milestone-check.
This commit adds a more descriptive docs about the caching process. In
addition, the `github.job` is now removed from the cache key
construction to allow jobs sharing the same cache. In addition, given we
now use `inputs.key-prefix`, the `github.job` is no longer relevant as
the `key-prefix` already creates a unique name for the cache.
We also update the lint and check commits job to use the same cache from
the unit test, given they are running in the same environment.
Turns out that actions/setup-go starting with @v4 also adds caching.
With that, our cache size on disk has almost doubled, leading to the
GitHub runner running out of space in certain situation.
We fix that by disabling the automated caching since we already have our
own, custom-tailored version.
This is a tradeoff of disk space (and with that cache size) and
compilation speed. Because we're still running into disk full errors
with the full build cache, we remove it for the cross compile step.
Which means we'll do more work each time.
To avoid build caches getting larger and larger, causing our builds to
fail, we separate the caches of the different job types.
If we don't separate them, then the caches can end up being a
combination of the different jobs, which isn't useful (for example the
build cache of the cross compilation build isn't useful in an
integration test and vice versa).
To avoid the problem where multiple jobs try to create the same cache
entry, we add the GitHub job ID to the cache key.
The two restore keys make it possible that a job can also restore a
cache entry from another job if none for the current job exist yet.
With this commit we allow the Go version that is set up to be
configurable and not dependent on a specific environment variable. This
will allow us to eventually extract the action into a tooling
repository.
For some reason we used to override the GOCACHE and GOPATH variables
earlier. This now causes the updated cache action not to pick up any
caches. As the overrides shouldn't be needed anymore, we remove them.
ziggie