Warning, /containers/docs/building-locally.md is written in an unsupported language. File is not indexed.
0001 # Building Locally
0002
0003 This guide explains how to build the EIC containers locally, taking advantage of available caches to speed up builds.
0004
0005 ## Prerequisites
0006
0007 - Docker with Buildx support (Docker 19.03+)
0008 - At least 100GB of free disk space
0009 - 8GB+ RAM recommended
0010
0011 ```bash
0012 # Verify Docker Buildx is available
0013 docker buildx version
0014 ```
0015
0016 ## Quick Start
0017
0018 ### Building the Base Image
0019
0020 ```bash
0021 cd /path/to/containers
0022
0023 # Build the base image
0024 docker buildx build \
0025 -f containers/debian/Dockerfile \
0026 -t debian_stable_base:local \
0027 containers/debian
0028 ```
0029
0030 ### Building the EIC Image
0031
0032 ```bash
0033 # Build the full XL environment
0034 docker buildx build \
0035 -f containers/eic/Dockerfile \
0036 --build-context spack-environment=spack-environment \
0037 --build-arg DOCKER_REGISTRY=ghcr.io/eic/ \
0038 --build-arg BUILDER_IMAGE=debian_stable_base \
0039 --build-arg RUNTIME_IMAGE=debian_stable_base \
0040 --build-arg INTERNAL_TAG=latest \
0041 --build-arg ENV=xl \
0042 -t eic_xl:local \
0043 containers/eic
0044 ```
0045
0046 ## Taking Advantage of Caching
0047
0048 ### 1. Registry Cache (Docker Layer Cache)
0049
0050 The public buildcache stored on ghcr.io can significantly speed up builds:
0051
0052 ```bash
0053 # Build with registry cache from ghcr.io
0054 docker buildx build \
0055 -f containers/debian/Dockerfile \
0056 --cache-from type=registry,ref=ghcr.io/eic/buildcache:debian_stable_base-master-amd64 \
0057 -t debian_stable_base:local \
0058 containers/debian
0059 ```
0060
0061 For the EIC image:
0062
0063 ```bash
0064 docker buildx build \
0065 -f containers/eic/Dockerfile \
0066 --build-context spack-environment=spack-environment \
0067 --build-arg DOCKER_REGISTRY=ghcr.io/eic/ \
0068 --build-arg BUILDER_IMAGE=debian_stable_base \
0069 --build-arg RUNTIME_IMAGE=debian_stable_base \
0070 --build-arg INTERNAL_TAG=latest \
0071 --build-arg ENV=xl \
0072 --cache-from type=registry,ref=ghcr.io/eic/buildcache:eic_xl-default-master-amd64 \
0073 -t eic_xl:local \
0074 containers/eic
0075 ```
0076
0077 ### 2. Spack Buildcache (Pre-built Binaries)
0078
0079 The most significant speedup comes from using pre-built Spack binaries. The containers are configured to automatically fetch from the public buildcache:
0080
0081 ```
0082 ghcr.io/eic/spack-v2025.07.0 # OCI-based Spack buildcache
0083 binaries.spack.io/v1.0 # Official Spack buildcache
0084 ```
0085
0086 No additional configuration is needed - the base image is pre-configured to use these mirrors.
0087
0088 ### 3. Local Build Caches
0089
0090 For repeated local builds, set up persistent caches:
0091
0092 ```bash
0093 # Create cache directories
0094 mkdir -p ~/.cache/eic-containers/{apt,spack,ccache}
0095
0096 # Build with local cache mounts
0097 docker buildx build \
0098 -f containers/debian/Dockerfile \
0099 --build-arg jobs=8 \
0100 -t debian_stable_base:local \
0101 containers/debian
0102 ```
0103
0104 ## Build Architecture Diagram
0105
0106 ```mermaid
0107 flowchart TB
0108 subgraph "Cache Sources"
0109 RC[ghcr.io Registry Cache<br/>Docker layers]
0110 BC[ghcr.io Spack Buildcache<br/>Pre-built binaries]
0111 LC[Local Cache<br/>ccache, apt]
0112 end
0113
0114 subgraph "Build Process"
0115 D[Dockerfile]
0116 D --> L1[Restore cache layers]
0117 L1 --> L2[Install packages]
0118 L2 --> L3[Configure Spack]
0119 L3 --> L4[Install from buildcache]
0120 L4 --> L5[Build remaining packages]
0121 L5 --> I[Final Image]
0122 end
0123
0124 RC --> L1
0125 BC --> L4
0126 LC --> L2
0127 LC --> L5
0128 ```
0129
0130 ## Full Build Script
0131
0132 Here's a complete script for building locally with all optimizations:
0133
0134 ```bash
0135 #!/bin/bash
0136 set -e
0137
0138 REPO_DIR="$(pwd)"
0139 REGISTRY="ghcr.io/eic"
0140 ARCH=$(uname -m | sed 's/x86_64/amd64/' | sed 's/aarch64/arm64/')
0141
0142 # Load version information
0143 source spack.sh
0144 source spack-packages.sh
0145 source key4hep-spack.sh
0146 source eic-spack.sh
0147
0148 # Helper to resolve git refs
0149 resolve_ref() {
0150 .ci/resolve_git_ref "$1" "$2"
0151 }
0152
0153 # Build base image
0154 echo "=== Building debian_stable_base ==="
0155 docker buildx build \
0156 -f containers/debian/Dockerfile \
0157 --build-arg SPACK_ORGREPO=${SPACK_ORGREPO} \
0158 --build-arg SPACK_VERSION=${SPACK_VERSION} \
0159 --build-arg SPACKPACKAGES_ORGREPO=${SPACKPACKAGES_ORGREPO} \
0160 --build-arg SPACKPACKAGES_VERSION=${SPACKPACKAGES_VERSION} \
0161 --build-arg KEY4HEPSPACK_ORGREPO=${KEY4HEPSPACK_ORGREPO} \
0162 --build-arg KEY4HEPSPACK_VERSION=${KEY4HEPSPACK_VERSION} \
0163 --build-arg EICSPACK_ORGREPO=${EICSPACK_ORGREPO} \
0164 --build-arg EICSPACK_VERSION=${EICSPACK_VERSION} \
0165 --build-arg jobs=8 \
0166 --cache-from type=registry,ref=${REGISTRY}/buildcache:debian_stable_base-master-${ARCH} \
0167 -t debian_stable_base:local \
0168 --load \
0169 containers/debian
0170
0171 # Build EIC image
0172 echo "=== Building eic_xl ==="
0173 docker buildx build \
0174 -f containers/eic/Dockerfile \
0175 --build-context spack-environment=spack-environment \
0176 --build-arg DOCKER_REGISTRY="" \
0177 --build-arg BUILDER_IMAGE=debian_stable_base:local \
0178 --build-arg RUNTIME_IMAGE=debian_stable_base:local \
0179 --build-arg INTERNAL_TAG="" \
0180 --build-arg ENV=xl \
0181 --cache-from type=registry,ref=${REGISTRY}/buildcache:eic_xl-default-master-${ARCH} \
0182 -t eic_xl:local \
0183 --load \
0184 containers/eic
0185
0186 echo "=== Build complete ==="
0187 echo "Run with: docker run -it eic_xl:local"
0188 ```
0189
0190 ## Build Arguments Reference
0191
0192 ### Base Image (containers/debian/Dockerfile)
0193
0194 | Argument | Description | Default |
0195 |----------|-------------|---------|
0196 | `BASE_IMAGE` | Base Debian image | `debian:stable-slim` |
0197 | `SPACK_ORGREPO` | Spack GitHub org/repo | `spack/spack` |
0198 | `SPACK_VERSION` | Spack version/branch | `develop` |
0199 | `SPACK_SHA` | Specific commit SHA | (resolved from version) |
0200 | `SPACK_CHERRYPICKS` | Space-separated cherry-pick SHAs | |
0201 | `SPACKPACKAGES_*` | Similar args for spack-packages | |
0202 | `KEY4HEPSPACK_*` | Similar args for key4hep-spack | |
0203 | `EICSPACK_*` | Similar args for eic-spack | |
0204 | `jobs` | Parallel build jobs | `1` |
0205
0206 ### EIC Image (containers/eic/Dockerfile)
0207
0208 | Argument | Description | Default |
0209 |----------|-------------|---------|
0210 | `DOCKER_REGISTRY` | Registry prefix for base images | `eicweb/` |
0211 | `BUILDER_IMAGE` | Builder base image name | `debian_stable_base` |
0212 | `RUNTIME_IMAGE` | Runtime base image name | `debian_stable_base` |
0213 | `INTERNAL_TAG` | Tag for base images | `master` |
0214 | `ENV` | Environment type (`ci` or `xl`) | `xl` |
0215 | `EDM4EIC_SHA` | Custom edm4eic commit | |
0216 | `EICRECON_SHA` | Custom eicrecon commit | |
0217 | `EPIC_SHA` | Custom epic commit | |
0218 | `JUGGLER_SHA` | Custom juggler commit | |
0219
0220 ## Troubleshooting
0221
0222 ### Build Fails with Out of Memory
0223
0224 Reduce the number of parallel jobs:
0225
0226 ```bash
0227 docker buildx build --build-arg jobs=2 ...
0228 ```
0229
0230 ### Build Takes Too Long
0231
0232 1. Ensure you're using registry cache (`--cache-from`)
0233 2. Check that Spack buildcache is accessible
0234 3. Build the CI environment first (smaller): `--build-arg ENV=ci`
0235
0236 ### Cannot Pull from ghcr.io
0237
0238 The buildcache is public, but if you encounter authentication issues:
0239
0240 ```bash
0241 # Anonymous pull should work
0242 docker pull ghcr.io/eic/debian_stable_base:latest
0243
0244 # Or login (for push access)
0245 echo $GITHUB_TOKEN | docker login ghcr.io -u $GITHUB_USER --password-stdin
0246 ```
0247
0248 ### Spack Installation Timeouts
0249
0250 The containers increase the default timeout. For local builds, ensure stable network connectivity.
0251
0252 ## Building for ARM64 on x86_64
0253
0254 Using QEMU emulation (slower but works):
0255
0256 ```bash
0257 # Setup QEMU
0258 docker run --privileged --rm tonistiigi/binfmt --install arm64
0259
0260 # Build for ARM64
0261 docker buildx build \
0262 --platform linux/arm64 \
0263 -f containers/debian/Dockerfile \
0264 -t debian_stable_base:local-arm64 \
0265 containers/debian
0266 ```
0267
0268 ## Next Steps
0269
0270 - See [Architecture Overview](architecture.md) for understanding the build structure
0271 - See [Build Pipeline](build-pipeline.md) for CI/CD details
0272 - See [Spack Environment](spack-environment.md) for package configuration
