Warning, /detector_benchmarks/README.md is written in an unsupported language. File is not indexed.
0001 ePIC Detector Benchmarks
0002 ========================
0003
0004 [](https://eicweb.phy.anl.gov/EIC/benchmarks/detector_benchmarks/-/commits/master)
0005
0006 ## Overview
0007
0008 Detector benchmarks are meant to provide a maintained set of performance plots for individual detector subsystems.
0009
0010 ## Documentation
0011
0012 - See [tutorial](https://eic.github.io/tutorial-developing-benchmarks/).
0013
0014 ## Adding new benchmarks
0015
0016 To get an idea of what to do look at an existing benchmark in the
0017 [`benchmarks` directory](https://github.com/eic/detector_benchmarks/tree/master/benchmarks).
0018 Currently a good reference for Snakemake instrumentation is available in the `tracking\_performances` benchmark.
0019 It relies on single particle simulations that can be either produced on eicweb or downloaded from official campagins.
0020
0021 ### File organization
0022
0023 For a minimal benchmark you'll need to add
0024 `benchmarks/<benchmark_name_here>/config.yml` and
0025 `benchmarks/<benchmark_name_here>/Snakemake`, plus the analysis script/macro.
0026 The `Snakefile` has to be included in the root `./Snakefile` of the repository.
0027 That common entry point is needed to ensure that common simulation samples can
0028 be defined to be re-used by several benchmarks at a time.
0029 The `config.yml` will require an include from the `./.gitlab-ci.yml`.
0030
0031 ### Pass/Fail tests
0032
0033 - Create a script that returns exit status 0 for success.
0034 - Any non-zero value will be considered failure.
0035
0036 ### Caching (`cache: True`) and rule `params`
0037
0038 When using Snakemake's `cache: True`, be aware that the cache key (the
0039 "provenance hash") is computed from:
0040
0041 - the rule's **unformatted** shell command template — i.e. the literal text
0042 `{wildcards.FOO}`, not the actual substituted value,
0043 - the job's `params`,
0044 - the content hash of input files that are *not* produced by another job,
0045 - the (recursively computed) provenance hash of upstream jobs, for inputs
0046 that *are* produced by another job.
0047
0048 As a consequence, wildcards that are only used in the `shell:` command or in
0049 `input:`/`output:` paths, and are not otherwise reflected in `params` or in
0050 the content of a non-generated input file, do **not** affect the cache key.
0051 This can cause jobs with different wildcard values (e.g. different
0052 particles/energies) to incorrectly collide onto the same cache entry.
0053
0054 To avoid this, any wildcard that affects the result of a cached rule (or of
0055 an upstream rule it depends on) must be listed explicitly in `params`, e.g.
0056 `PARTICLE = lambda wildcards: wildcards.PARTICLE`.
