Warning, /acts/CI/ccache-snapshots.md is written in an unsupported language. File is not indexed.
0001 # CERN S3 compiler caches
0002
0003 Builds, Analysis, Detray, Traccc, and PyPI wheels restore local ccache snapshots,
0004 including direct-mode manifests. Restores are anonymous. Main pushes publish;
0005 PyPI additionally permits scheduled/manual nightly builds of main. Explicit
0006 release-ref wheel builds are read-only. Writers use the protected `s3-sccache`
0007 environment with `S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY`; readers need no
0008 environment.
0009
0010 Storage: `https://s3.cern.ch`, region `cern`, bucket `cache`. Prefixes under
0011 `acts-sccache/ccache-snapshots/<repository>/` separate workflows, jobs, platforms,
0012 and matrix variants. The original `linux_ubuntu/v1/` prefix is unchanged.
0013 Each new variant starts cold until its first main publication.
0014
0015 Restore and publish remain explicit so jobs retain saves after failures and
0016 before later tests. Cancelled jobs skip cleanup and publication. LCG nightly only publishes after a successful configure.
0017 Cleanup and statistics run where ccache is available: the job container, sourced
0018 LCG/Key4hep view, EIC Docker container, or wheel build environment. Wheels clean
0019 before tests; failures before that point rely on ccache's automatic size cleanup.
0020 Cache limits are 2 GB, with 4 GB for `linux_ubuntu`, Analysis, Detray
0021 container CPU Debug, and Traccc Debug/RelWithDebInfo builds.
0022 Traccc CUDA Debug caches only C++ compilation so its PTX check still gets
0023 fresh intermediate files; other CUDA and HIP builds also cache device compilation.
0024
0025 Publication uploads an immutable tar, then updates `latest.json`. There is no
0026 extra concurrency group; overlapping uploads may leave an older valid snapshot
0027 as latest. Restores verify checksums and extract into staging. Transfer failures
0028 are nonfatal. Entries are already compressed, so the tar is uncompressed. The
0029 nightly `ccache cleanup` workflow deletes non-latest archives without an age
0030 grace period, preserving each variant's `latest.json` and its archive. Manual
0031 runs default to dry-run. Missing/invalid pointers skip that variant and fail the
0032 cleanup job; deletion failures also fail the job. Credentials need S3 list and
0033 delete access. Logs and the job summary report deleted and retained space.
0034 A concurrent restore may lose its archive and fall back to a cold build.
0035 Remove the existing S3 expiration rule after verifying cleanup, so it cannot
0036 expire a rarely used variant's latest archive or pointer.
0037
0038 Compare GitHub's restore, build, and publish step durations and ccache statistics.
0039 Do not set `CCACHE_BASEDIR`: path rewriting breaks source-path FPE masks.
0040
0041 Job logs include a compact `CCACHE_STATS ` JSON record before explicit cleanup.
0042 The reporter checks jq and ccache JSON support; failures warn without emitting
0043 a measurement or failing the build. EIC and Key4hep counters cover both build
0044 phases. Wheels report inside their build environment before tests; a wheel
0045 compilation/repair failure before that hook produces no record. Missing records
0046 must not be treated as zero.
