Warning, /containers/docs/spack-environment.md is written in an unsupported language. File is not indexed.
0001 # Spack Environment
0002
0003 The EIC containers use [Spack](https://spack.io/) for package management. This document explains the Spack configuration and how packages are selected and built.
0004
0005 ## Spack Repository Stack
0006
0007 The container uses multiple Spack repositories with the following priority (highest first):
0008
0009 ```mermaid
0010 flowchart TB
0011 subgraph "Repository Priority"
0012 direction TB
0013 E[eic/eic-spack<br/>EIC-specific packages]
0014 K[key4hep/key4hep-spack<br/>High-energy physics packages]
0015 SP[spack/spack-packages<br/>Extended community packages]
0016 S[spack/spack<br/>Core Spack packages]
0017 end
0018
0019 E --> K --> SP --> S
0020
0021 subgraph "Locations in Container"
0022 E -.- EL[/opt/spack-packages/repos/eic-spack]
0023 K -.- KL[/opt/spack-packages/repos/key4hep-spack]
0024 SP -.- SPL[/opt/spack-packages/repos/spack_repo/builtin]
0025 S -.- SL[/opt/spack/var/spack/repos/builtin]
0026 end
0027 ```
0028
0029 ## Environment Configurations
0030
0031 ### Environment Files
0032
0033 ```
0034 spack-environment/
0035 ├── packages.yaml # Global package preferences
0036 ├── concretizer.yaml # Concretization settings
0037 ├── view.yaml # Spack view configuration
0038 ├── ci/ # CI environment
0039 │ ├── spack.yaml # CI package specs
0040 │ └── epic/
0041 │ └── spack.yaml # CI epic-specific specs
0042 └── xl/ # Full (XL) environment
0043 ├── spack.yaml # XL package specs
0044 └── epic/
0045 └── spack.yaml # XL epic-specific specs
0046 ```
0047
0048 ### packages.yaml Structure
0049
0050 The `packages.yaml` file defines version preferences and build variants:
0051
0052 ```yaml
0053 packages:
0054 all:
0055 require:
0056 - '%gcc' # Use GCC compiler
0057 - any_of: [+ipo, '@:'] # Enable IPO if supported
0058 - any_of: [build_system=cmake, '@:']
0059 - any_of: [build_type=Release, '@:']
0060
0061 # Package-specific requirements
0062 root:
0063 require:
0064 - '@6.36.04'
0065 - cxxstd=20 +fftw +fortran +gdml +http +python +root7 +tmva
0066 - any_of: [+opengl +webgui, -opengl -webgui]
0067 ```
0068
0069 ### Environment Variants
0070
0071 #### CI Environment (`spack-environment/ci/`)
0072
0073 Minimal environment for continuous integration testing:
0074
0075 ```yaml
0076 specs:
0077 - acts
0078 - dd4hep -ddeve # Without event display
0079 - geant4 -opengl # Without OpenGL
0080 - root -opengl -webgui # Without GUI components
0081 - ...
0082 ```
0083
0084 **Characteristics:**
0085 - Headless operation (no GUI dependencies)
0086 - Faster build times
0087 - Smaller image size
0088 - Essential reconstruction tools only
0089
0090 #### XL Environment (`spack-environment/xl/`)
0091
0092 Full development environment with all features:
0093
0094 ```yaml
0095 specs:
0096 - acts
0097 - dd4hep +ddeve # With event display
0098 - geant4 +opengl # With OpenGL support
0099 - root +opengl +webgui # Full GUI support
0100 - claude-code # AI coding assistant
0101 - github-copilot # AI coding assistant
0102 - emacs # Editor
0103 - jupyter # Notebooks
0104 - py-tensorflow # Machine learning
0105 - py-torch # Machine learning
0106 - ...
0107 ```
0108
0109 **Additional packages in XL:**
0110 - Development tools (emacs, gdb, valgrind, lcov)
0111 - GUI support (Qt, OpenGL, X11)
0112 - Machine learning (TensorFlow, PyTorch, ONNX)
0113 - Jupyter notebook ecosystem
0114 - Additional Monte Carlo generators
0115
0116 ### EPIC Subenvironment
0117
0118 Both CI and XL have an `epic/` subdirectory with the EPIC detector packages:
0119
0120 ```yaml
0121 spack:
0122 include_concrete:
0123 - /opt/spack-environment/xl # or ci
0124 specs:
0125 - algorithms
0126 - edm4eic
0127 - eicrecon
0128 - epic@main # Current development
0129 - epic@25.08.0 # Tagged versions
0130 - epic@25.09.0
0131 - epic@25.10.0
0132 - juggler
0133 ```
0134
0135 ## Version Control
0136
0137 ### Version Configuration Files
0138
0139 | File | Contents |
0140 |------|----------|
0141 | `spack.sh` | Spack core version |
0142 | `spack-packages.sh` | Spack-packages version |
0143 | `key4hep-spack.sh` | Key4HEP-spack version |
0144 | `eic-spack.sh` | EIC-spack version |
0145
0146 ### Cherry-picks
0147
0148 Spack versions can include cherry-picks from newer commits:
0149
0150 ```bash
0151 # From spack.sh
0152 SPACK_VERSION="v1.0.2"
0153 SPACK_CHERRYPICKS=$(cat <<- EOF
0154 09f75ee426a2e05e0543570821582480ff823ba5
0155 a462612b64e97fa7dfe461c32c58553fd6ec63c5
0156 EOF
0157 )
0158 ```
0159
0160 ## Package Selection Flow
0161
0162 ```mermaid
0163 flowchart TB
0164 subgraph "Version Selection"
0165 P[packages.yaml<br/>Version preferences]
0166 E[Environment spack.yaml<br/>Package specs]
0167 C[Concretizer<br/>Dependency resolution]
0168 end
0169
0170 P --> C
0171 E --> C
0172 C --> R[Resolved Package Set]
0173
0174 subgraph "Installation"
0175 R --> BC{In Buildcache?}
0176 BC -->|Yes| BI[Install from binary]
0177 BC -->|No| BS[Build from source]
0178 end
0179 ```
0180
0181 ## Concretizer Configuration
0182
0183 From `concretizer.yaml`:
0184
0185 ```yaml
0186 concretizer:
0187 targets:
0188 host_compatible: false
0189 unify: true
0190 reuse: false
0191 ```
0192
0193 - **`host_compatible: false`**: Build for target architecture, not host
0194 - **`unify: true`**: Single version of each package in environment
0195 - **`reuse: false`**: Don't reuse packages from buildcache during concretization
0196
0197 ## Key Packages
0198
0199 ### Core Physics Packages
0200
0201 | Package | Description |
0202 |---------|-------------|
0203 | `root` | Data analysis framework |
0204 | `geant4` | Particle simulation |
0205 | `dd4hep` | Detector description toolkit |
0206 | `acts` | Track reconstruction |
0207 | `hepmc3` | Event record format |
0208 | `podio` | Event data model framework |
0209 | `edm4hep` | Common HEP event data model |
0210
0211 ### EIC-Specific Packages
0212
0213 | Package | Description |
0214 |---------|-------------|
0215 | `edm4eic` | EIC event data model |
0216 | `eicrecon` | EIC reconstruction framework |
0217 | `epic` | EPIC detector geometry |
0218 | `npsim` | Nuclear physics simulation |
0219 | `juggler` | Gaudi algorithms for EIC |
0220 | `irt` | Imaging RICH Toolkit |
0221
0222 ### Development Tools
0223
0224 | Package | Description |
0225 |---------|-------------|
0226 | `cmake` | Build system |
0227 | `ninja` | Fast build tool |
0228 | `gdb` | Debugger |
0229 | `valgrind` | Memory analyzer |
0230 | `lcov` | Code coverage |
0231 | `iwyu` | Include-what-you-use |
0232
0233 ## Modifying Packages
0234
0235 ### Updating a Package Version
0236
0237 1. Edit `spack-environment/packages.yaml`:
0238 ```yaml
0239 packages:
0240 mypackage:
0241 require:
0242 - '@1.2.3' # Update version
0243 ```
0244
0245 2. If package definition needs updating (from newer Spack):
0246 - Add cherry-pick to `spack-packages.sh`
0247
0248 ### Adding a New Package
0249
0250 1. Check if package exists in Spack repositories
0251 2. Add to appropriate `spack.yaml`:
0252 ```yaml
0253 specs:
0254 - new-package
0255 ```
0256 3. Add version/variant requirements to `packages.yaml` if needed
0257
0258 ### Creating Custom Package Versions
0259
0260 Using custom commit SHA:
0261
0262 ```bash
0263 # Via workflow dispatch inputs
0264 EDM4EIC_VERSION=abc123def...
0265
0266 # In Dockerfile, this becomes:
0267 edm4eic@git.abc123def=main
0268 ```
0269
0270 ## Inspecting the Environment
0271
0272 Inside a container:
0273
0274 ```bash
0275 # List installed packages
0276 spack find
0277
0278 # Show package info
0279 spack info root
0280
0281 # Show package dependencies
0282 spack spec root
0283
0284 # Show environment graph
0285 spack graph --dot
0286
0287 # Show loaded environment
0288 spack env status
0289 ```
0290
0291 ## Related Documentation
0292
0293 - [Architecture Overview](architecture.md) - Build system structure
0294 - [Building Locally](building-locally.md) - Local build instructions
0295 - [Build Pipeline](build-pipeline.md) - CI workflow details