containers/docs/architecture.md

0001 # Architecture Overview
0002
0003 The EIC container infrastructure uses a multi-stage build approach with separate builder and runtime tracks. This design ensures efficient builds while producing lightweight final images.
0004
0005 ## Build Strategy
0006
0007 The container build follows a two-track approach:
0008
0009 ```mermaid
0010 flowchart TB
0011     subgraph "Builder Track"
0012         A[builder_image<br/>debian_stable_base] --> B[builder_concretization_default<br/>Concretize spack environment]
0013         B --> C[builder_installation_default<br/>Build packages]
0014         C --> D[builder_concretization_custom<br/>Concretize custom versions]
0015         D --> E[builder_installation_custom<br/>Build custom packages]
0016     end
0017
0018     subgraph "Runtime Track"
0019         F[runtime_image<br/>debian_stable_base] --> G[runtime_concretization_default<br/>Copy spack.lock from builder]
0020         G --> H[runtime_installation_default<br/>Install from buildcache]
0021         H --> I[runtime_concretization_custom<br/>Copy custom spack.lock]
0022         I --> J[runtime_installation_custom<br/>Install custom from buildcache]
0023         J --> K[Final Image<br/>eic_ci / eic_xl]
0024     end
0025
0026     C -.->|spack.lock| G
0027     C -.->|buildcache| H
0028     E -.->|spack.lock| I
0029     E -.->|buildcache| J
0030 ```
0031
0032 ## Multi-Architecture Support
0033
0034 The infrastructure supports both `amd64` and `arm64` architectures through parallel builds:
0035
0036 ```mermaid
0037 flowchart LR
0038     subgraph "Build Phase"
0039         A1[Build amd64]
0040         A2[Build arm64]
0041     end
0042
0043     subgraph "Manifest Phase"
0044         M[Create Multi-Arch Manifest]
0045     end
0046
0047     A1 --> D1[amd64 digest]
0048     A2 --> D2[arm64 digest]
0049     D1 --> M
0050     D2 --> M
0051     M --> R[Registry<br/>ghcr.io/eic/*]
0052 ```
0053
0054 ## Container Layer Structure
0055
0056 ### Base Image (debian_stable_base)
0057
0058 ```mermaid
0059 graph TB
0060     subgraph "debian_stable_base"
0061         D[Debian Stable Slim]
0062         D --> P1[System Packages<br/>git, curl, compilers, etc.]
0063         P1 --> G[GCC + Clang Compilers]
0064         G --> S[Spack Installation]
0065         S --> SR1[spack/spack repository]
0066         S --> SR2[spack/spack-packages repository]
0067         S --> SR3[key4hep/key4hep-spack repository]
0068         S --> SR4[eic/eic-spack repository]
0069         SR4 --> M[Spack Mirrors<br/>binaries.spack.io, ghcr.io/eic/spack-*]
0070     end
0071 ```
0072
0073 ### EIC Image (eic_ci / eic_xl)
0074
0075 ```mermaid
0076 graph TB
0077     subgraph "EIC Container"
0078         B[debian_stable_base]
0079         B --> E1[Default Spack Environment<br/>ci or xl]
0080         E1 --> E2[Custom Environment<br/>epic, eicrecon, etc.]
0081         E2 --> F[Final Configuration]
0082         F --> BM[Benchmarks]
0083         F --> CP[Campaigns]
0084         F --> SC[Scripts & Entrypoint]
0085     end
0086 ```
0087
0088 ## Spack Repository Hierarchy
0089
0090 The Spack package definitions come from multiple repositories with priority ordering:
0091
0092 ```mermaid
0093 flowchart TB
0094     subgraph "Priority Order (High to Low)"
0095         E[eic/eic-spack<br/>EIC-specific packages]
0096         K[key4hep/key4hep-spack<br/>Key4HEP packages]
0097         SP[spack/spack-packages<br/>Community packages]
0098     end
0099
0100     E --> K --> SP
0101
0102     P[Package Resolution] --> E
0103 ```
0104
0105 ## Caching Architecture
0106
0107 Multiple caching layers are used to optimize build times:
0108
0109 ```mermaid
0110 flowchart TB
0111     subgraph "Build Cache Types"
0112         RC[Registry Cache<br/>Docker layer cache in ghcr.io]
0113         GC[GitHub Actions Cache<br/>ccache, apt, spack source]
0114         BC[Spack Buildcache<br/>Pre-built binaries on ghcr.io]
0115     end
0116
0117     subgraph "Cache Locations"
0118         RC --> R1[ghcr.io/eic/buildcache:*]
0119         GC --> G1[ccache]
0120         GC --> G2["/var/cache/apt"]
0121         GC --> G3["/var/cache/spack"]
0122         BC --> B1[ghcr.io/eic/spack-v2025.07.0]
0123     end
0124 ```
0125
0126 ## Environment Variants
0127
0128 ### CI Environment (eic_ci)
0129
0130 Minimal environment for continuous integration:
0131 - Core HEP packages (ROOT, Geant4, DD4hep)
0132 - Essential reconstruction tools
0133 - No GUI dependencies (`-opengl`, `-webgui`)
0134
0135 ### XL Environment (eic_xl)
0136
0137 Full development environment:
0138 - All CI packages plus GUI support
0139 - Development tools (emacs, gdb, valgrind, etc.)
0140 - Additional Python packages
0141 - Machine learning tools (TensorFlow, PyTorch, ONNX)
0142 - Jupyter notebook support
0143
0144 ## Version Configuration
0145
0146 Package versions are controlled through several configuration files:
0147
0148 | File | Purpose |
0149 |------|---------|
0150 | `spack.sh` | Spack core version and cherry-picks |
0151 | `spack-packages.sh` | Spack-packages version and cherry-picks |
0152 | `key4hep-spack.sh` | Key4HEP-spack version |
0153 | `eic-spack.sh` | EIC-spack version |
0154 | `spack-environment/packages.yaml` | Package version preferences and variants |