Warning, /EICrecon/docs/howto/make_plugin.md is written in an unsupported language. File is not indexed.
0001 # JANA plugins in EICrecon
0002
0003 Here are instructions for creating your own user plugin that can create
0004 custom histograms/trees that will be added to the output ROOT histograms
0005 file. This covers just the most common use-case of making a plugin with
0006 a _JEventProcessor_.
0007
0008 These instructions can be used to build a plugin that is kept outside of
0009 the `EICrecon` source tree, but that compiles against it. Thus, if you have
0010 access to a read-only `EICrecon` build, you can still use it to make a plugin
0011 that can install to a directory you do have write access to.
0012
0013 ## EICrecon_MY
0014 The `EICrecon_MY` environment variable is useful for building plugins outside
0015 the `EICrecon` source tree. Set this to a directory where you have write
0016 permission. The build instructions below will install the plugin to that
0017 directory. When `eicrecon` is run, it will also look for plugins in the
0018 `$EICrecon_MY` directory (as well as the central `EICrecon` build you are using.)
0019
0020 ~~~bash
0021 mkdir EICrecon_MY
0022 export EICrecon_MY=${PWD}/EICrecon_MY
0023 ~~~
0024
0025
0026 ## Creating a plugin with _eicmkplugin.py_
0027 EICrecon installs a script called _eicmkplugin.py_ that will generate all
0028 of the boilerplate code for a new plugin. This includes a directory, a
0029 _CMakeLists.txt_ file, and some skeleton code. The instructions below create
0030 a plugin called _DaveTest_ and then compile it. It will be installed in
0031 the directory pointed to by your `EICrecon_MY` environment variable (if set).
0032
0033 ~~~bash
0034 eicmkplugin.py DaveTest
0035 cmake -S DaveTest -B DaveTest/build
0036 cmake --build DaveTest/build --target install
0037 ~~~
0038
0039 You can test that the plugin installed and can load correctly by running
0040 `eicrecon` with it.
0041
0042 ~~~bash
0043 eicrecon -Pplugins=DaveTest,JTest -Pjana:nevents=10
0044 ~~~
0045
0046 <details>
0047 <summary>Click here to see full output of above command</summary>
0048
0049 ~~~
0050 ____ _ ___ ___ _
0051 `MM' dM. `MM\ `M' dM.
0052 MM ,MMb MMM\ M ,MMb
0053 MM d'YM. M\MM\ M d'YM. ____
0054 MM ,P `Mb M \MM\ M ,P `Mb 6MMMMb
0055 MM d' YM. M \MM\ M d' YM. MM' `Mb
0056 MM ,P `Mb M \MM\ M ,P `Mb ,MM
0057 MM d' YM. M \MM\M d' YM. ,MM'
0058 (8) MM ,MMMMMMMMb M \MMM ,MMMMMMMMb ,M'
0059 (( ,M9 d' YM. M \MM d' YM. ,M'
0060 YMMMM9 _dM_ _dMM_M_ \M _dM_ _dMM_MMMMMMMM
0061
0062 [INFO] Creating pipe named "/tmp/jana_status" for status info.
0063 [INFO] Setting signal handlers
0064 [INFO] JPluginLoader: Initializing plugin "/work/eic2/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon_MY/plugins/DaveTest.so"
0065 [INFO] JPluginLoader: Initializing plugin "/usr/local/plugins/JTest.so"
0066 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/podio.so"
0067 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/dd4hep.so"
0068 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/acts.so"
0069 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/log.so"
0070 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/rootfile.so"
0071 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/BEMC.so"
0072 [INFO] JPluginLoader: Initializing plugin "/w/eic-scshelf2104/users/davidl/2022.08.29.eic-shell/EICwork/EICrecon/lib/EICrecon/plugins/EEMC.so"
0073 [INFO] JArrowProcessingController: NUMA Configuration
0074
0075 Affinity strategy: compute-bound (favor fewer hyperthreads)
0076 Locality strategy: global
0077 Location count: 1
0078 +--------+----------+-------+--------+-----------+--------+
0079 | worker | location | cpu | core | numa node | socket |
0080 +--------+----------+-------+--------+-----------+--------+
0081 | 0 | 0 | 0 | 0 | 0 | 0 |
0082 | 1 | 0 | 1 | 1 | 0 | 0 |
0083 | 2 | 0 | 2 | 2 | 0 | 0 |
0084 | 3 | 0 | 3 | 3 | 1 | 0 |
0085 | 4 | 0 | 4 | 4 | 1 | 0 |
0086 | 5 | 0 | 5 | 5 | 0 | 0 |
0087 | 6 | 0 | 6 | 6 | 0 | 0 |
0088 | 7 | 0 | 7 | 7 | 1 | 0 |
0089 | 8 | 0 | 8 | 8 | 1 | 0 |
0090 | 9 | 0 | 9 | 9 | 1 | 0 |
0091 | 10 | 0 | 10 | 10 | 0 | 0 |
0092 | 11 | 0 | 11 | 11 | 0 | 0 |
0093 | 12 | 0 | 12 | 12 | 0 | 0 |
0094 | 13 | 0 | 13 | 13 | 1 | 0 |
0095 | 14 | 0 | 14 | 14 | 1 | 0 |
0096 | 15 | 0 | 15 | 15 | 0 | 0 |
0097 | 16 | 0 | 16 | 16 | 0 | 0 |
0098 | 17 | 0 | 17 | 17 | 1 | 0 |
0099 | 18 | 0 | 18 | 18 | 1 | 0 |
0100 | 19 | 0 | 19 | 19 | 1 | 0 |
0101 | 20 | 0 | 20 | 20 | 2 | 1 |
0102 | 21 | 0 | 21 | 21 | 2 | 1 |
0103 | 22 | 0 | 22 | 22 | 2 | 1 |
0104 | 23 | 0 | 23 | 23 | 3 | 1 |
0105 | 24 | 0 | 24 | 24 | 3 | 1 |
0106 | 25 | 0 | 25 | 25 | 2 | 1 |
0107 | 26 | 0 | 26 | 26 | 2 | 1 |
0108 | 27 | 0 | 27 | 27 | 3 | 1 |
0109 | 28 | 0 | 28 | 28 | 3 | 1 |
0110 | 29 | 0 | 29 | 29 | 3 | 1 |
0111 | 30 | 0 | 30 | 30 | 2 | 1 |
0112 | 31 | 0 | 31 | 31 | 2 | 1 |
0113 | 32 | 0 | 32 | 32 | 2 | 1 |
0114 | 33 | 0 | 33 | 33 | 3 | 1 |
0115 | 34 | 0 | 34 | 34 | 3 | 1 |
0116 | 35 | 0 | 35 | 35 | 2 | 1 |
0117 | 36 | 0 | 36 | 36 | 2 | 1 |
0118 | 37 | 0 | 37 | 37 | 3 | 1 |
0119 | 38 | 0 | 38 | 38 | 3 | 1 |
0120 | 39 | 0 | 39 | 39 | 3 | 1 |
0121 | 40 | 0 | 40 | 0 | 0 | 0 |
0122 | 41 | 0 | 41 | 1 | 0 | 0 |
0123 | 42 | 0 | 42 | 2 | 0 | 0 |
0124 | 43 | 0 | 43 | 3 | 1 | 0 |
0125 | 44 | 0 | 44 | 4 | 1 | 0 |
0126 | 45 | 0 | 45 | 5 | 0 | 0 |
0127 | 46 | 0 | 46 | 6 | 0 | 0 |
0128 | 47 | 0 | 47 | 7 | 1 | 0 |
0129 | 48 | 0 | 48 | 8 | 1 | 0 |
0130 | 49 | 0 | 49 | 9 | 1 | 0 |
0131 | 50 | 0 | 50 | 10 | 0 | 0 |
0132 | 51 | 0 | 51 | 11 | 0 | 0 |
0133 | 52 | 0 | 52 | 12 | 0 | 0 |
0134 | 53 | 0 | 53 | 13 | 1 | 0 |
0135 | 54 | 0 | 54 | 14 | 1 | 0 |
0136 | 55 | 0 | 55 | 15 | 0 | 0 |
0137 | 56 | 0 | 56 | 16 | 0 | 0 |
0138 | 57 | 0 | 57 | 17 | 1 | 0 |
0139 | 58 | 0 | 58 | 18 | 1 | 0 |
0140 | 59 | 0 | 59 | 19 | 1 | 0 |
0141 | 60 | 0 | 60 | 20 | 2 | 1 |
0142 | 61 | 0 | 61 | 21 | 2 | 1 |
0143 | 62 | 0 | 62 | 22 | 2 | 1 |
0144 | 63 | 0 | 63 | 23 | 3 | 1 |
0145 | 64 | 0 | 64 | 24 | 3 | 1 |
0146 | 65 | 0 | 65 | 25 | 2 | 1 |
0147 | 66 | 0 | 66 | 26 | 2 | 1 |
0148 | 67 | 0 | 67 | 27 | 3 | 1 |
0149 | 68 | 0 | 68 | 28 | 3 | 1 |
0150 | 69 | 0 | 69 | 29 | 3 | 1 |
0151 | 70 | 0 | 70 | 30 | 2 | 1 |
0152 | 71 | 0 | 71 | 31 | 2 | 1 |
0153 | 72 | 0 | 72 | 32 | 2 | 1 |
0154 | 73 | 0 | 73 | 33 | 3 | 1 |
0155 | 74 | 0 | 74 | 34 | 3 | 1 |
0156 | 75 | 0 | 75 | 35 | 2 | 1 |
0157 | 76 | 0 | 76 | 36 | 2 | 1 |
0158 | 77 | 0 | 77 | 37 | 3 | 1 |
0159 | 78 | 0 | 78 | 38 | 3 | 1 |
0160 | 79 | 0 | 79 | 39 | 3 | 1 |
0161 +--------+----------+-------+--------+-----------+--------+
0162
0163 [INFO] Configuration Parameters
0164 ----------------------------
0165 Name Value
0166 ------------ --------------
0167 csv:dest_dir .
0168 jana:nevents 10
0169 plugins DaveTest,JTest
0170 ----------------------------
0171
0172 [INFO] Component Summary
0173
0174 SOURCES
0175 -----------------------------------
0176 Plugin Name Source
0177 -------- ----------- ------------
0178 JTest.so JTestParser dummy_source
0179 -----------------------------------
0180 PROCESSORS
0181 ---------------------------------------
0182 Plugin Name
0183 ----------- --------------------------
0184 DaveTest.so DaveTestProcessor
0185 JTest.so JTestPlotter
0186 JTest.so JCsvWriter<JTestTrackData>
0187 ---------------------------------------
0188 FACTORIES
0189 --------------------------------------------------------------------
0190 Plugin Object name Tag
0191 -------- -------------------------- ------------------------------
0192 JTest.so JTestEventData
0193 JTest.so JTestTrackData
0194 EEMC.so edm4eic::ProtoCluster EcalEndcapNIslandProtoClusters
0195 EEMC.so edm4eic::ProtoCluster EcalEndcapNTruthProtoClusters
0196 EEMC.so edm4eic::CalorimeterHit EcalEndcapNRecHits
0197 EEMC.so edm4eic::Cluster EcalEndcapNClusters
0198 EEMC.so edm4eic::Cluster EcalEndcapNMergedClusters
0199 BEMC.so edm4hep::RawCalorimeterHit EcalBarrelNRawHits
0200 EEMC.so edm4hep::RawCalorimeterHit EcalEndcapNRawHits
0201 --------------------------------------------------------------------
0202
0203 [INFO] Starting processing with 1 threads requested...
0204 [INFO] JArrowProcessingController: run(): Launching 1 workers
0205 [INFO] JArrow: Initializing JEventProcessor 'DaveTestProcessor'
0206 [INFO] JArrow: Initializing JEventProcessor 'JTestPlotter'
0207 [INFO] JArrow: Initializing JEventProcessor 'JCsvWriter<JTestTrackData>'
0208 [INFO] JArrow: Initializing JEventSource 'dummy_source' (JTestParser)
0209 [INFO] Status: 5 events processed 10.0 Hz (10.0 Hz avg)
0210 [INFO] JArrow: Finalizing JEventSource 'dummy_source' (JTestParser)
0211 [INFO] Status: 10 events processed 13.2 Hz (11.4 Hz avg)
0212 [INFO] All workers have stopped.
0213 [INFO] Merging threads ...
0214 [INFO] JArrow: Finalizing JEventProcessor 'DaveTestProcessor'
0215 [INFO] JArrow: Finalizing JEventProcessor 'JTestPlotter'
0216 [INFO] JArrow: Finalizing JEventProcessor 'JCsvWriter<JTestTrackData>'
0217 [INFO] Event processing ended.
0218 [INFO] JArrowProcessingController: Final Report
0219 Thread team size [count]: 1
0220 Total uptime [s]: 0.8802
0221 Uptime delta [s]: 0.3781
0222 Completed events [count]: 10
0223 Inst throughput [Hz]: 13.2
0224 Avg throughput [Hz]: 11.4
0225 Sequential bottleneck [Hz]: 212
0226 Parallel bottleneck [Hz]: 12.1
0227 Efficiency [0..1]: 0.94
0228
0229 +--------------------------+------------+--------+-----+---------+-------+--------+---------+-------------+
0230 | Name | Status | Type | Par | Threads | Chunk | Thresh | Pending | Completed |
0231 +--------------------------+------------+--------+-----+---------+-------+--------+---------+-------------+
0232 | dummy_source | Finished | Source | F | 0 | 40 | - | - | 10 |
0233 | processors | Finished | Sink | T | 0 | 1 | 80 | 0 | 10 |
0234 +--------------------------+------------+--------+-----+---------+-------+--------+---------+-------------+
0235 +--------------------------+-------------+--------------+----------------+--------------+----------------+
0236 | Name | Avg latency | Inst latency | Queue latency | Queue visits | Queue overhead |
0237 | | [ms/event] | [ms/event] | [ms/visit] | [count] | [0..1] |
0238 +--------------------------+-------------+--------------+----------------+--------------+----------------+
0239 | dummy_source | 4.73 | 3.16 | 0.000174 | 11 | 4.05e-05 |
0240 | processors | 82.8 | 88.4 | 0.000324 | 50 | 1.96e-05 |
0241 +--------------------------+-------------+--------------+----------------+--------------+----------------+
0242 +----+----------------------+-------------+------------+-----------+----------------+------------------+
0243 | ID | Last arrow name | Useful time | Retry time | Idle time | Scheduler time | Scheduler visits |
0244 | | | [ms] | [ms] | [ms] | [ms] | [count] |
0245 +----+----------------------+-------------+------------+-----------+----------------+------------------+
0246 | 0 | idle | 0.0357 | 0 | 0 | 0.000538 | 21 |
0247 +----+----------------------+-------------+------------+-----------+----------------+------------------+
0248
0249 ~~~
0250 </details>
0251
0252 Note that in the above command, a second plugin _JTest_ which supplies dummy events and
0253 the _jana_nevents_ configuration parameter is set to only process 10 events.
0254
0255 For the plugin to be useful, you will need to edit the files in the `DaveTest`
0256 directory. There are two files: _DaveTestProcessor.cc_ and a _DaveTestProcessor.h_.
0257 Start with the _DaveTestProcessor.h_ file. Edit your file to look similar to the one
0258 below. You will need to add include lines for each data type you are interested
0259 in (see ```#include <edm4hep/Cluster.h>```), a Prefetch line for each collection
0260 type, and then pointer for every histogram or tree you wish to create.
0261
0262 ```cpp
0263 #include <JANA/JEventProcessorSequentialRoot.h>
0264 #include <TH2D.h>
0265 #include <TFile.h>
0266
0267 #include <edm4hep/Cluster.h>
0268
0269 class DaveTestProcessor: public JEventProcessorSequentialRoot {
0270 private:
0271
0272 // Declare histogram and tree pointers here. e.g.
0273 TH1D* hClusterEnergy = nullptr;
0274
0275 public:
0276 DaveTestProcessor() { SetTypeName(NAME_OF_THIS); }
0277
0278 void InitWithGlobalRootLock() override;
0279 void ProcessSequential(const std::shared_ptr<const JEvent>& event) override;
0280 void FinishWithGlobalRootLock() override;
0281 };
0282 ```
0283 Next, edit the _DaveTest.cc_ file to look similar to what is below. The key things
0284 you will need to add are the lines in the `InitWithGlobalRootLock` and
0285 `ProcessSequential` methods. See the comments in the`InitWithGlobalRootLock`
0286 method for an explanation of those.
0287
0288 The one line in the `ProcessSequential` method will loop over the values
0289 in the `clusters` member. As a `Prefetch` object, _JANA_ will automatically
0290 call all algorithms needed to produce the `Cluster` objects prior to calling
0291 `ProcessSequential`. This works for objects created by algorithms or ones
0292 coming straight from the file.
0293
0294 ```cpp
0295 #include "DaveTestProcessor.h"
0296 #include <services/rootfile/RootFile_service.h>
0297
0298 // The following just makes this a JANA plugin
0299 extern "C" {
0300 void InitPlugin(JApplication *m_app) {
0301 InitJANAPlugin(m_app);
0302 m_app->Add(new DaveTestProcessor);
0303 }
0304 }
0305
0306 //-------------------------------------------
0307 // InitWithGlobalRootLock
0308 //-------------------------------------------
0309 void DaveTestProcessor::InitWithGlobalRootLock(){
0310
0311 // This ensures the histograms created here show up in the same root file as
0312 // other plugins attached to the process. Place them in dedicated directory
0313 // to avoid name conflicts.
0314 auto rootfile_svc = GetApplication()->GetService<RootFile_service>();
0315 auto rootfile = rootfile_svc->GetHistFile();
0316 rootfile->mkdir("DaveTest")->cd();
0317
0318 hClusterEnergy = new TH1D("hClusterEnergy", "EcalEndcapNClusters energy (GeV)", 200, 0, 0.075);
0319 }
0320
0321 //-------------------------------------------
0322 // ProcessSequential
0323 //-------------------------------------------
0324 void DaveTestProcessor::ProcessSequential(const std::shared_ptr<const JEvent>& event) {
0325 const auto &clusters = *static_cast<const edm4eic::ClusterCollection*>(event->GetCollectionBase("EcalEndcapNClusters"));
0326
0327 for (auto cluster : clusters) hClusterEnergy->Fill(cluster.getEnergy());
0328 }
0329
0330 //-------------------------------------------
0331 // FinishWithGlobalRootLock
0332 //-------------------------------------------
0333 void DaveTestProcessor::FinishWithGlobalRootLock() {
0334
0335 // Do any final calculations here.
0336 }
0337 ```
0338
0339 You can now test this using you favorite simulated data file or with the one in the
0340 following commands:
0341
0342 ~~~bash
0343 wget https://eicaidata.s3.amazonaws.com/2022-09-04_pgun_e-_podio-0.15_edm4hep-0.6_0-30GeV_alldir_1k.edm4hep.root
0344 eicrecon -Pplugins=DaveTest 2022-09-04_pgun_e-_podio-0.15_edm4hep-0.6_0-30GeV_alldir_1k.edm4hep.root
0345 ~~~
0346
0347 This should run through the events filling you histogram resulting in the file
0348 `eicrecon.root`. The histogram should look something like this:
0349
0350 
0351
0352
0353
0354 ## Manually creating a plugin
0355 For people who really love to get their fingers dirty and feel the above
0356 instructions are just too easy, here are some more detailed instructions
0357 for creating a plugin from scratch without `eicmkplugin.py`.
0358
0359 There is a copy/paste CMake file that should automatically create plugin out of sources.
0360
0361 - plugin name is taken from a directory name
0362 - there should be /</plugin name/>/.cc file with `void InitPlugin(JApplication *m_app)` function
0363
0364
0365 ### Create a plugin:
0366
0367 E.g. if you want to create a plugin named `my_plugin`
0368
0369 - Create a directory `my_plugin`
0370 - Create a file `my_plugin.cc` which will have `InitPlugin` function
0371 - Create `CMakeLists.txt` with the content below
0372 - Add all others files (cmake GLOB is used)
0373
0374 ### Recommended cmake:
0375
0376 Recommended CMake for a plugin:
0377
0378 ```cmake
0379 cmake_minimum_required(VERSION 3.16)
0380
0381 # Automatically set plugin name the same as the directory name
0382 # Don't forget string(REPLACE " " "_" PLUGIN_NAME ${PLUGIN_NAME}) if this dir has spaces in its name
0383 get_filename_component(PLUGIN_NAME ${CMAKE_CURRENT_LIST_DIR} NAME)
0384
0385 # Function creates ${PLUGIN_NAME}_plugin and ${PLUGIN_NAME}_library targets
0386 # Setting default includes, libraries and installation paths
0387 plugin_add(${PLUGIN_NAME})
0388
0389 # Find dependencies
0390 find_package(JANA REQUIRED)
0391 find_package(EDM4HEP REQUIRED)
0392 find_package(podio REQUIRED)
0393 find_package(DD4hep REQUIRED)
0394 find_package(ROOT REQUIRED COMPONENTS Core Tree Hist RIO EG)
0395
0396 # The macro grabs sources as *.cc *.cpp *.c and headers as *.h *.hh *.hpp
0397 # Then correctly sets sources for ${_name}_plugin and ${_name}_library targets
0398 # Adds headers to the correct installation directory
0399 plugin_glob_all(${PLUGIN_NAME})
0400
0401 # Add include directories
0402 # (same as target_include_directories but for both plugin and library)
0403 plugin_include_directories(${PLUGIN_NAME} SYSTEM PUBLIC ${JANA_INCLUDE_DIR} ${podio_INCLUDE_DIR} ${EDM4HEP_INCLUDE_DIR} ${DD4hep_INCLUDE_DIRS} ${ROOT_INCLUDE_DIRS})
0404
0405 # Add libraries
0406 # (same as target_include_directories but for both plugin and library)
0407 plugin_link_libraries(${PLUGIN_NAME} ${JANA_LIB})
0408 ```
0409
0410 ### CMake macros:
0411
0412 There are `plugin_...` macros that are slim wrappers trying to minimize an amount of boilerplate
0413 code of each plugin cmake scripts. Macros mimic CMake functions like `target_link_libraries` => `plugin_link_libraries`.
