tutorial-setting-up-environment/_episodes/02-eic-shell.md

0001 ---
0002 title: "The EIC Software Environment"
0003 teaching: 5
0004 exercises: 5
0005 questions:
0006 - "What is the EIC environment?"
0007 objectives:
0008 - "Understand what we mean with the EIC environment, or `eic-shell`."
0009 - "Appreciate the benefits of containers as development environment."
0010 keypoints:
0011 - "The EIC environment `eic-shell` is a singularity/docker container with a curated selection of software components."
0012 ---
0013 ## Why a EIC standard environment?
0014 - EIC software is modular, so not just a matter of a single `make install` command but close to 20 necessary dependencies and 10 EIC-specific packages.
0015 - Laboratory operating systems are stable and (therefore) slow to upgrade to new versions, even though we need those versions in the modern software we write:
0016   - gcc-11 or gcc-12 for C++17 ranges, C++20 for concepts and filtered ranges, and C++23 for zipped ranges
0017   - underlying dependencies like ACTS, DD4hep, podio, etc, each have many options, some of which are important to set right (or you end up with a DD4hep that doesn't do some of the things we need it to do)
0018 - The standard environment gives all users access to exactly the same environment that is used for benchmarks, productions, and continuous integration on the github and gitlab servers.
0019 - The standard environments are versioned and can be retrieved (or rebuilt) at a later time in case you wish to revisit an old result. Specifying the specific version of the standard environment you used for a particular plot will definitively determine what versions of e.g. geant4 and ROOT you used.
0020 - If you encounter a bug that you can reproduce in a standard environment, it makes it much much easier for software developers to fix. Making it easier for developers to fix you bug will typically result in your bug getting fixed quicker. If developers cannot reproduce the bug, it often becomes a fishing expedition that few developers have appetite for.
0021
0022 ## Are you required to use the EIC standard environment?
0023 - No. You can use any environment you want. We are merely trying to make this easy for all users to get started, in particular if you are not yet familiar with compiling source code from scratch.
0024 - Most of the people in the software groups use a mix of the standard environment and other installations (for example, we may have ROOT installed separately).
0025 - We may ask you to reproduce issues that you observe in the standard environment before we can act on them.
0026 - If there is a specific workflow or use case that you cannot achieve in the standard environment, let us know and we'll work with you to address that.
0027
0028 ## How to get `eic-shell`, the EIC standard environment, and what it is?
0029 - The EIC standard environment is accessed with the `eic-shell` command. That will start an `eic-shell` session in the standard environment.
0030 - To install the `eic-shell` command in the directory `~/eic/` (which you may need to create), you can run the following command:
0031 ```console
0032 cd ~/eic
0033 curl --location https://get.epic-eic.org | bash
0034 ```
0035 - You can also save the file at https://get.epic-eic.org as `install.sh` and run this script by hand:
0036 ```console
0037 cd ~/eic
0038 wget --output-document install.sh https://get.epic-eic.org
0039 bash install.sh
0040 ```
0041 - The install script will search for several components that are required (see the prerequisites):
0042   - which operating system you are running (linux and mac are supported),
0043   - whether you have singularity/apptainer or docker installed (see prerequisites),
0044   - whether you have access to the Cern VM file system at `/cvmfs` (typically available on clusters, and can be easily installed on individual linux systems, see https://cernvm.cern.ch/fs/).
0045 - If you are on a system that requires downloading the full container (up to 13 GB), this may take a while and no progress bar is shown.
0046 - The install script will create a new file `./eic-shell` which will start the `eic-shell` environment, i.e. the EIC standard environment.
0047
0048 > Note: An alternative location for downloading this script is at https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/install.sh, in case you experience difficulties accessing the custom domain.
0049 {: .callout}
0050
0051 > Note: On some high performance computing servers, there are security restrictions that require the use of additional flags `curl --insecure` and `wget --no-check-certificate`.
0052 {: .callout}
0053
0054 > Exercise 1:
0055 > - Install the `eic-shell` executable in a suitable location on the system you will be using mainly for EIC work.
0056 > - Look at the output and compare with what is on the instructor's screen. If you do not have a `/cvmfs` directory on your system, this may take a while as the environment has to be downloaded. Keep this in mind for later, as you will have to perform updates periodically.
0057 > - Advanced users: Take a look at the installation options when you run `bash install.sh --help`. The include information on how to tweak your setup for specific use scenarios.
0058 {: .challenge}
0059
0060 ## How to start `eic-shell` on common systems?
0061 You can start `eic-shell` from inside the directory where it is installed with
0062 ```console
0063 cd ~/eic
0064 ./eic-shell
0065 ```
0066 or you can start it from any directory with `~/eic/eic-shell` (if you used the same directory structure).
0067 - When you have successfully started `eic-shell` you should see a `jug_dev> ` prompt, indicating that you are now **inside** the EIC standard environment.
0068 - Pro-tip: You can also add the directory where `eic-shell` is installed to your search path, so you can start `eic-shell` from anywhere. Or you can install `eic-shell` in a directory that is already in your search path.
0069
0070 > Exercise:
0071 > - Run the `eic-shell` script from the current directory. You should get the `jug_dev> ` prompt. Exit the environment again.
0072 > - Take a look at the running options with `eic-shell --help`. If you do not have a `/cvmfs` directory on your system, you will need to run `eic-shell --upgrade` periodically to ensure that you are staying up to date.
0073 > - Advanced users: Set up your environment to be able to run `eic-shell` from anywhere.
0074 {: .challenge}
0075
0076 ## What is where in the eic-shell environment?
0077 When you start `eic-shell`, you enter into a container: a self-contained operating system that integrates with the core of the operating system on the host system. When you look at the top-level directories, you'll see that they are similar to the directories on the host system. Don't let that fool you: you are looking at content that is only inside the container.
0078
0079 There are a few directories that are shared between the container and the host operating system: the current directory you are working on, your home directories, etc. Essentially, the typical directories where you would store your own work (as opposed to system directories).
0080
0081 All programs that are installed in the container, are accessible under the `/opt/local` directory tree. This is a standard location for programs that are not installed by the operating system's package manager, which is how we install the dependencies for the EIC software stack. It also means that these programs are automatically found by the `eic-shell` without needing to modify the search path.
0082
0083 The jumble of files in `/opt/local` are in fact merely a view into software installed in a more organized fashion in `/opt/software`, potentially for different compilers, operating systems, etc. The full definition of all software installed in `/opt/software` is in `/opt/spack-environment/dev/spack.yaml`, the ultimate definition of the EIC standard environment.
0084
0085 > Exercise:
0086 > - Using `cd` and `ls`, check that you can navigate to various directories in the container, and that you can access the directories for your own work.
0087 > - Verify that you can run some of the software dependencies that are installed in `eic-shell`. For example, run the command `ddsim --help` to get some informaton on one way to run Geant4 simulations on DD4hep-based geometries. Use the command `which ddsim` to see where `ddsim` is located.
0088 > - Advanced users: Based on the location of `ddsim` found in the previous step, verify that this is a symbolic link to a different location under `/opt/software`. Verify that the version in the path of that location agrees with the version specified in the `spack.yaml` file.
0089 {: .challenge}
0090
0091 ## Notes on starting graphical programs
0092 Graphical programs work just as well inside the `eic-shell` environment as outside, with the exception of docker on Mac where some additional work is needed. On a Mac it is therefore often easier to open ROOT files outside the container.
0093
0094 {% include links.md %}