Back to home page

EIC code displayed by LXR

 
 

    


Warning, /containers/docs/spack-environment.md is written in an unsupported language. File is not indexed.

0001 # Spack Environment
0002 
0003 The EIC containers use [Spack](https://spack.io/) for package management. This document explains the Spack configuration and how packages are selected and built.
0004 
0005 ## Spack Repository Stack
0006 
0007 The container uses multiple Spack repositories with the following priority (highest first):
0008 
0009 ```mermaid
0010 flowchart TB
0011     subgraph "Repository Priority"
0012         direction TB
0013         E[eic/eic-spack<br/>EIC-specific packages]
0014         K[key4hep/key4hep-spack<br/>High-energy physics packages]
0015         SP[spack/spack-packages<br/>Extended community packages]
0016         S[spack/spack<br/>Core Spack packages]
0017     end
0018     
0019     E --> K --> SP --> S
0020     
0021     subgraph "Locations in Container"
0022         E -.- EL[/opt/spack-packages/repos/eic-spack]
0023         K -.- KL[/opt/spack-packages/repos/key4hep-spack]
0024         SP -.- SPL[/opt/spack-packages/repos/spack_repo/builtin]
0025         S -.- SL[/opt/spack/var/spack/repos/builtin]
0026     end
0027 ```
0028 
0029 ## Environment Configurations
0030 
0031 ### Environment Files
0032 
0033 ```
0034 spack-environment/
0035 ├── packages.yaml      # Global package preferences
0036 ├── concretizer.yaml   # Concretization settings
0037 ├── view.yaml          # Spack view configuration
0038 ├── ci/                # CI environment
0039 │   ├── spack.yaml     # CI package specs
0040 │   └── epic/
0041 │       └── spack.yaml # CI epic-specific specs
0042 └── xl/                # Full (XL) environment
0043     ├── spack.yaml     # XL package specs
0044     └── epic/
0045         └── spack.yaml # XL epic-specific specs
0046 ```
0047 
0048 ### packages.yaml Structure
0049 
0050 The `packages.yaml` file defines version preferences and build variants:
0051 
0052 ```yaml
0053 packages:
0054   all:
0055     require:
0056     - '%gcc'                         # Use GCC compiler
0057     - any_of: [+ipo, '@:']           # Enable IPO if supported
0058     - any_of: [build_system=cmake, '@:']
0059     - any_of: [build_type=Release, '@:']
0060   
0061   # Package-specific requirements
0062   root:
0063     require:
0064     - '@6.36.04'
0065     - cxxstd=20 +fftw +fortran +gdml +http +python +root7 +tmva
0066     - any_of: [+opengl +webgui, -opengl -webgui]
0067 ```
0068 
0069 ### Environment Variants
0070 
0071 #### CI Environment (`spack-environment/ci/`)
0072 
0073 Minimal environment for continuous integration testing:
0074 
0075 ```yaml
0076 specs:
0077 - acts
0078 - dd4hep -ddeve           # Without event display
0079 - geant4 -opengl          # Without OpenGL
0080 - root -opengl -webgui    # Without GUI components
0081 - ...
0082 ```
0083 
0084 **Characteristics:**
0085 - Headless operation (no GUI dependencies)
0086 - Faster build times
0087 - Smaller image size
0088 - Essential reconstruction tools only
0089 
0090 #### XL Environment (`spack-environment/xl/`)
0091 
0092 Full development environment with all features:
0093 
0094 ```yaml
0095 specs:
0096 - acts
0097 - dd4hep +ddeve           # With event display
0098 - geant4 +opengl          # With OpenGL support
0099 - root +opengl +webgui    # Full GUI support
0100 - claude-code             # AI coding assistant
0101 - github-copilot          # AI coding assistant
0102 - emacs                   # Editor
0103 - jupyter                 # Notebooks
0104 - py-tensorflow           # Machine learning
0105 - py-torch                # Machine learning
0106 - ...
0107 ```
0108 
0109 **Additional packages in XL:**
0110 - Development tools (emacs, gdb, valgrind, lcov)
0111 - GUI support (Qt, OpenGL, X11)
0112 - Machine learning (TensorFlow, PyTorch, ONNX)
0113 - Jupyter notebook ecosystem
0114 - Additional Monte Carlo generators
0115 
0116 ### EPIC Subenvironment
0117 
0118 Both CI and XL have an `epic/` subdirectory with the EPIC detector packages:
0119 
0120 ```yaml
0121 spack:
0122   include_concrete:
0123   - /opt/spack-environment/xl  # or ci
0124   specs:
0125   - algorithms
0126   - edm4eic
0127   - eicrecon
0128   - epic@main              # Current development
0129   - epic@25.08.0           # Tagged versions
0130   - epic@25.09.0
0131   - epic@25.10.0
0132   - juggler
0133 ```
0134 
0135 ## Version Control
0136 
0137 ### Version Configuration Files
0138 
0139 | File | Contents |
0140 |------|----------|
0141 | `spack.sh` | Spack core version |
0142 | `spack-packages.sh` | Spack-packages version |
0143 | `key4hep-spack.sh` | Key4HEP-spack version |
0144 | `eic-spack.sh` | EIC-spack version |
0145 
0146 ### Cherry-picks
0147 
0148 Spack versions can include cherry-picks from newer commits:
0149 
0150 ```bash
0151 # From spack.sh
0152 SPACK_VERSION="v1.0.2"
0153 SPACK_CHERRYPICKS=$(cat <<- EOF
0154 09f75ee426a2e05e0543570821582480ff823ba5
0155 a462612b64e97fa7dfe461c32c58553fd6ec63c5
0156 EOF
0157 )
0158 ```
0159 
0160 ## Package Selection Flow
0161 
0162 ```mermaid
0163 flowchart TB
0164     subgraph "Version Selection"
0165         P[packages.yaml<br/>Version preferences]
0166         E[Environment spack.yaml<br/>Package specs]
0167         C[Concretizer<br/>Dependency resolution]
0168     end
0169     
0170     P --> C
0171     E --> C
0172     C --> R[Resolved Package Set]
0173     
0174     subgraph "Installation"
0175         R --> BC{In Buildcache?}
0176         BC -->|Yes| BI[Install from binary]
0177         BC -->|No| BS[Build from source]
0178     end
0179 ```
0180 
0181 ## Concretizer Configuration
0182 
0183 From `concretizer.yaml`:
0184 
0185 ```yaml
0186 concretizer:
0187   targets:
0188     host_compatible: false
0189   unify: true
0190   reuse: false
0191 ```
0192 
0193 - **`host_compatible: false`**: Build for target architecture, not host
0194 - **`unify: true`**: Single version of each package in environment
0195 - **`reuse: false`**: Don't reuse packages from buildcache during concretization
0196 
0197 ## Key Packages
0198 
0199 ### Core Physics Packages
0200 
0201 | Package | Description |
0202 |---------|-------------|
0203 | `root` | Data analysis framework |
0204 | `geant4` | Particle simulation |
0205 | `dd4hep` | Detector description toolkit |
0206 | `acts` | Track reconstruction |
0207 | `hepmc3` | Event record format |
0208 | `podio` | Event data model framework |
0209 | `edm4hep` | Common HEP event data model |
0210 
0211 ### EIC-Specific Packages
0212 
0213 | Package | Description |
0214 |---------|-------------|
0215 | `edm4eic` | EIC event data model |
0216 | `eicrecon` | EIC reconstruction framework |
0217 | `epic` | EPIC detector geometry |
0218 | `npsim` | Nuclear physics simulation |
0219 | `juggler` | Gaudi algorithms for EIC |
0220 | `irt` | Imaging RICH Toolkit |
0221 
0222 ### Development Tools
0223 
0224 | Package | Description |
0225 |---------|-------------|
0226 | `cmake` | Build system |
0227 | `ninja` | Fast build tool |
0228 | `gdb` | Debugger |
0229 | `valgrind` | Memory analyzer |
0230 | `lcov` | Code coverage |
0231 | `iwyu` | Include-what-you-use |
0232 
0233 ## Modifying Packages
0234 
0235 ### Updating a Package Version
0236 
0237 1. Edit `spack-environment/packages.yaml`:
0238    ```yaml
0239    packages:
0240      mypackage:
0241        require:
0242        - '@1.2.3'  # Update version
0243    ```
0244 
0245 2. If package definition needs updating (from newer Spack):
0246    - Add cherry-pick to `spack-packages.sh`
0247 
0248 ### Adding a New Package
0249 
0250 1. Check if package exists in Spack repositories
0251 2. Add to appropriate `spack.yaml`:
0252    ```yaml
0253    specs:
0254    - new-package
0255    ```
0256 3. Add version/variant requirements to `packages.yaml` if needed
0257 
0258 ### Creating Custom Package Versions
0259 
0260 Using custom commit SHA:
0261 
0262 ```bash
0263 # Via workflow dispatch inputs
0264 EDM4EIC_VERSION=abc123def...
0265 
0266 # In Dockerfile, this becomes:
0267 edm4eic@git.abc123def=main
0268 ```
0269 
0270 ## Inspecting the Environment
0271 
0272 Inside a container:
0273 
0274 ```bash
0275 # List installed packages
0276 spack find
0277 
0278 # Show package info
0279 spack info root
0280 
0281 # Show package dependencies
0282 spack spec root
0283 
0284 # Show environment graph
0285 spack graph --dot
0286 
0287 # Show loaded environment
0288 spack env status
0289 ```
0290 
0291 ## Related Documentation
0292 
0293 - [Architecture Overview](architecture.md) - Build system structure
0294 - [Building Locally](building-locally.md) - Local build instructions
0295 - [Build Pipeline](build-pipeline.md) - CI workflow details