Back to home page

EIC code displayed by LXR

 
 

    


Warning, /DD4hep/doc/usermanuals/DDCond/DDCondManual.tex is written in an unsupported language. File is not indexed.

0001 %=============================================================================
0002 \documentclass[10pt,a4paper]{article}
0003 %
0004 \input{./setup/DD4hep-setup.tex}
0005 \input{./setup/AIDA2020-setup.tex}
0006 %
0007 \pagestyle{fancyplain}{\fancyfoot[C]{\sffamily{DDCond User Manual}}}
0008 %
0009 \graphicspath{{./figs/}}
0010 \begin{document}   
0011 %
0012 \mytitle{DDCond}
0013 {
0014 Conditions Support for the \\
0015 \vspace{0.5cm}
0016 DD4hep Geometry Description \\
0017 \vspace{0.5cm}
0018 Toolkit
0019 \vspace{2cm}
0020 }
0021 {
0022 M. Frank \\
0023 {CERN, 1211 Geneva 23, Switzerland}
0024 }
0025 %
0026 %
0027 %==  Abstract  ===============================================================
0028 \pagestyle{plain}
0029 \pagenumbering{Roman}
0030 \setcounter{page}{1}
0031 \begin{abstract}
0032 %=============================================================================
0033 
0034 \noindent
0035 \normalsize
0036 Experimental setups in High Energy Physics are highly complex assemblies 
0037 consisting of various detector devices typically called {\it{subdetectors}}.
0038 To properly interpret the electronic signals which form the response of 
0039 particle collisions inside these subdetectors other auxiliary data are 
0040 necessary. These auxiliary data typically are time dependent - though normally
0041 at a much longer scale than the event data itself. The conditions part of the 
0042 \DDhep toolkit, called \DDC, addresses the management and the access of such
0043 conditions data. The Manual describes a solution, which pools groups of these
0044 time dependent data according to its validity. This approach firstly
0045 allows to quickly access all relevant data for a given particle collision.
0046 efficient caching mechansims and allows to quickly determine which data items
0047 need to be accessed from a persistent medium.
0048 The design is strongly driven by easy of use;
0049 developers of detector descriptions and applications using
0050 them should provide minimal information and minimal specific
0051 code to achieve the desired result.
0052 
0053 \end{abstract}
0054 
0055 \vspace{8cm}
0056 
0057 \begin{center}
0058 {\large{\bf{
0059 \begin{tabular} {| l | l | l |}
0060 \hline
0061 \multicolumn{3}{| c |}{} \\[0.2cm]
0062 \multicolumn{3}{| c |}{Document History} \\[0.2cm]
0063 \multicolumn{3}{| c |}{} \\[0.2cm]
0064 \hline
0065                  &      &        \\
0066 Document         &      &        \\
0067 version          & Date & Author \\[0.2cm] \hline
0068                  &      &        \\
0069 1.0              & 10/4/2014 & Markus Frank CERN/LHCb  \\
0070                  &      &        \\        \hline 
0071 \end{tabular}
0072 }}}
0073 \end{center}
0074 
0075 \clearpage
0076 %
0077 %
0078 %==  TOC  ====================================================================
0079 \tableofcontents
0080 \clearpage
0081 %
0082 %
0083 %=============================================================================
0084 % Manual
0085 %=============================================================================
0086 \pagenumbering{arabic}
0087 \setcounter{page}{1}
0088 
0089 %=============================================================================
0090 \section{Introduction}
0091 \label{sec:ddcond-user-manual-introduction}
0092 %=============================================================================
0093 \noindent
0094 In a high energy physics experiment the data originating from particle 
0095 collisions (so called $Event$-$data$) in most cases require supplementary, 
0096 mostly  environmental, calibration- or alignment data to extract the physics
0097 content from the recorded event data. These supplementary data are 
0098 time-dependent and  typically called $conditons$. The ability of an 
0099 experiment to produce correct and timely results depends on the complete 
0100 and efficient availability of  needed conditions for each stage of data 
0101 handling. This manual should introduce to the \DDC toolkit, which provides 
0102 access to conditions data within the \DDH data structures~\cite{bib:dd4hep-manual}.
0103 
0104 \noindent
0105 The \DDC extensions to the \DDH toolkit formalize both the access and 
0106 the management to time-dependent data necessary to process the event data.
0107 In this manual we will shortly describe the model used to organize and manage 
0108 the conditions data internally and then describe the user interface to
0109 actually perform the required actions.
0110 These conditions data typically are stored in a database. Nearly every
0111 high energy physics experiment has strong feelings how to efficiently
0112 read and store the conditions data in terms of tablespace organization 
0113 and data format.
0114 For this reason \DDC does not attempt to solve the persistency problem,
0115 but rather defines an interface used to load missing data items from the 
0116 persistent medium. Any persistent data representation, which can fulfill
0117 the requirements of this interface may be adopted by the \DDC 
0118 conditions caching and access mechanism.
0119 
0120 \noindent
0121 At the end of this manual a walk-through of an example is discussed to 
0122 illustrate, which steps have to be performed to use the \DDH conditions
0123 store within a client application.
0124 
0125 %=============================================================================
0126 \subsection{Definition of Conditions Data}
0127 \label{subsec:ddcond-conditions-data}
0128 %=============================================================================
0129 \noindent
0130 Conditions data are firstly
0131 \begin{itemize}\itemcompact
0132 \item raw data values. Raw data values are recorded from measurement 
0133       devices such as 
0134       thermometers, pressure devices or 
0135       geometrical parameters resulting
0136       from survey parameters and others.
0137       These data may change with time, but have one and only one version.
0138 \item Secondly there is the large group of data derived from the raw values.
0139       These derived values are transformed from one or several raw values
0140       into new data items with an interval of validity being the
0141       intersection of the intervals of validity of its ingredients.
0142       Effectively every raw measurement requires calibration to represent
0143       a useful value. Hence, nearly all raw values require such a 
0144       transformation. Since these data are re-calibrated, not only one version
0145       exists, but many e.g. as a result of different calibration algorithms. 
0146 \end{itemize}
0147 Typically one data processing application predefines for all events 
0148 to be processed the corresponding versions of the conditions data to be used.
0149 This time span typically is much larger than the intervals of validity 
0150 of single data value.
0151 The collection of all individual data item version for such a large time interval
0152 is called a "global tag".
0153 Within such a global tag, several conditions values of the same data item, but 
0154 with a different interval of validity may be accessed.
0155 
0156 \noindent
0157 Given this definition it is evident that the division between raw values 
0158 and derived conditions is rather fluent. Derived data as a result of
0159 a calibration process are technically identical to raw values in an
0160 analysis application using these re-calibrated constants.
0161 Hence, as soon as derived data enter the conditions database they are 
0162 technically identical to raw values.
0163 
0164 \noindent
0165 To support such calibration processes producing derived conditions 
0166 data, \DDC provides a mechanism to project new values given 
0167 a set of recipes provided by the user. This recipes can project a 
0168 set of coherent new conditions for a given event time accordingly.
0169 
0170 
0171 %=============================================================================
0172 \subsection{Conditions Slices}
0173 \label{subsec:ddcond-conditions-slices}
0174 %=============================================================================
0175 
0176 \noindent
0177 Conditions slices define a coherent set of conditions data valid for an event 
0178 recorded at a specific time. Each of the individual conditions of the slice
0179 has a certain interval of validity, hence the validity of the entire slice
0180 is defined as the intersection of these validities.
0181 As a corollary, the slice may be valid for more than one event as long as the
0182 event's time stamp is within this intersection. To maximize the flexibility,
0183 and to allow users to implement private slice implementations, slices have 
0184 a common interface, the $ConditionsMap$ (See section~\ref{subsec:ddcond-conditionsmap}). 
0185 For most practical purposes and to share tools between slice implementations, 
0186 this interface is sufficient.
0187 
0188 %=============================================================================
0189 \section{Generic Concepts and Design}
0190 \label{sec:ddcond-design-concepts}
0191 %=============================================================================
0192 
0193 \noindent 
0194 The \DDH conditions mechanism was designed to be very flexible concerning 
0195 back-end implementations. Most of the conditions and alignment utilities offered 
0196 by \DDH are available if a minimal interface is respected. This minimal interface
0197 includes a container called $ConditionsMap$ (See section~\ref{subsec:ddcond-conditionsmap})
0198 and the layout of the conditions objects (See section~\ref{subsec:ddcond-conditions-data}).
0199 The conditions objects contain a flexible user defined payload and a generic, 
0200 interface used to interact with tools and the generic container object or
0201 conditions slices as described in section~\ref{subsec:ddcond-conditions-slices}.
0202 
0203 %=============================================================================
0204 \subsection{Condition Objects and Conditions Data}
0205 \label{subsec:ddcond-conditions-data}
0206 %=============================================================================
0207 
0208 \noindent
0209 A conditions objects serves two purposes:
0210 \begin{itemize}\itemcompact
0211 \item Firstly, it supports the basic functionality which is generic to any 
0212       condition -- independent of the actual user payload. This information 
0213       includes access to the interval of validity and the key to uniquely identify
0214       the condition. 
0215 
0216 \item Secondly, the objects hosts and manages a user payload, the actual
0217       conditions data. These data are freely user defined. An automatic 
0218       parsing mechanism from a string representation is supported if the 
0219       payload-class can properly described using a boost::spirit
0220       parsing structure. Default type implementations are defined for 
0221       \begin{itemize}\itemcompact
0222         \item all primitive data types, 
0223         \item ROOT::Math::XYZVector, ROOT::Math::XYZPoint, ROOT::Math::PxPyPzEVector.
0224         \item a number of STL containers of the above basic data types:\\
0225                 std::vector\textless TYPE\textgreater, 
0226                 std::list\textless TYPE\textgreater,
0227                 std::set\textless TYPE\textgreater,\\
0228                 std::map\textless int,TYPE\textgreater,
0229                 std::map\textless string,TYPE\textgreater,\\
0230                 std::pair\textless int,TYPE\textgreater,
0231                 std::pair\textless string,TYPE\textgreater.
0232       \end{itemize}
0233       Additional types can easily be implemented using boost::spirit if the basic
0234       representation is in the form of a string. Dummy boost::spirit parsers 
0235       may be implemented if the conversion to and from strings is not required.
0236 \item Thirdly, it supports the basic functionality required by a 
0237       conditions management framework, which implements the $ConditionsMap$ 
0238       interface.
0239 \end{itemize}
0240 For completeness we include here the basic data access methods of the conditions
0241 class\\
0242 (see \detdesc{classdd4hep_1_1_condition.html}{DD4hep/Conditions.h}):
0243 
0244 \begin{unnumberedcode}
0245   class Condition: public Handle<detail::ConditionObject> {
0246     /** Interval of validity                                   */
0247     /// Access the IOV type
0248     const IOVType& iovType()  const;
0249     /// Access the IOV block
0250     const IOV& iov()  const;
0251  
0252     /** Conditions identification using integer keys.          */
0253     /// Hash identifier
0254     key_type key()  const;
0255     /// DetElement part of the identifier
0256     detkey_type  detector_key()  const;
0257     /// Item part of the identifier
0258     itemkey_type item_key()  const;
0259 
0260     /** Conditions meta-data and handling of the data binding  */
0261     /// Access the opaque data block
0262     OpaqueData& data()  const;
0263     /// Access to the type information
0264     const std::type_info& typeInfo() const;
0265     /// Access to the grammar type
0266     const BasicGrammar& descriptor() const;
0267     /// Check if object is already bound....
0268     bool is_bound()  const  {  return isValid() ? data().is_bound() : false;  }
0269     /// Bind the data of the conditions object to a given format.
0270     template <typename T> T& bind();
0271     /// Set and bind the data of the conditions object to a given format.
0272     template <typename T> T& bind(const std::string& val);
0273     /// Generic getter. Specify the exact type, not a polymorph type
0274     template <typename T> T& get();
0275     /// Generic getter (const version). Specify the exact type, not a polymorph type
0276     template <typename T> const T& get() const;
0277     ...
0278   };
0279 \end{unnumberedcode}
0280 Please be aware that the access to the IOV and the IOVType is only 
0281 possible if supported by the caching mechanism.
0282 
0283 \noindent
0284 Using the $OpaqueData$ data structure and its concrete implementation,
0285 the user can map any data item to the conditions object using the 
0286 $bind()$ method and retrieve the data back using $get()$.
0287 Clearly, the left should know what the right does and the types to be
0288 retrieved back must match be bound data types.
0289 
0290 \noindent
0291 The following code-snippet shows how to bind conditions data:
0292 \vspace{-0.2cm}
0293 \begin{unnumberedcode}
0294   Condition cond = ...;
0295   // Fill conditions data by hand:
0296   std::vector<int>& data = cond.bind<std::vector<int> >();
0297   data.push_back(0);
0298   data.push_back(1);    .....
0299 
0300   // Fill conditions data from the string representation using boost::spirit:
0301   std::string str = "[0,1,2]";
0302   std::vector<int>& data = cond.bind<std::vector<int> >(str);
0303   int i = data[0]; ....  
0304 \end{unnumberedcode}
0305 This is an example how to access already bound data:
0306 \vspace{-0.2cm}
0307 \begin{unnumberedcode}
0308   Condition cond = ...;
0309 
0310   // Fill conditions data by hand:
0311   std::vector<int>& data = cond.get<std::vector<int> >();
0312 \end{unnumberedcode}
0313 
0314 
0315 %=============================================================================
0316 \subsection{The ConditionsMap Interface}
0317 \label{subsec:ddcond-conditionsmap}
0318 %=============================================================================
0319 
0320 \noindent
0321 The $ConditionsMap$ interface (see 
0322 defines the lowest common denominator to
0323 allow tools or clients to interact with conditions of a given slice.
0324 This interface defines the interaction of clients with a conditions slice.
0325 These interactions cover both the data access and the data management 
0326 within a slice. The interface allows to
0327 \begin{itemize}\itemcompact
0328 \item access individual conditions by the detector element
0329 and a given item key. The interface allows
0330 \item to scan conditions according to the detector element or 
0331 \item to scan all conditions contained. Further it allows 
0332 \item insert conditions to the mapping and
0333 \item to clear the content.
0334 \end{itemize}
0335 The provision of these basic interaction mechanisms allows us to build
0336 very generic tools firstly for conditions, but also later for the
0337 management and th computation of alignment data as described in
0338 the \DDA manual~\cite{bib:ddalign-manual}.
0339 
0340 \noindent
0341 The $ConditionsMap$ interface class, which supports this basic functionality
0342 has the following entry points:
0343 \begin{unnumberedcode}
0344   class ConditionsMap   {
0345   public:
0346     /// Insert a new entry to the map. The detector element key and 
0347     /// the item key make a unique global conditions key
0348     virtual bool insert(DetElement         detector,
0349                         Condition::itemkey_type key,
0350                         Condition          condition) = 0;
0351     /// Interface to access conditions by hash value. The detector element key 
0352     /// and the item key make a unique global conditions key
0353     virtual Condition get(DetElement              detector,
0354                           Condition::itemkey_type key) const = 0;
0355     /// Interface to scan data content of the conditions mapping
0356     virtual void scan(const Condition::Processor& processor) const = 0;
0357 
0358     /// No ConditionsMap overload: Access all conditions within 
0359     /// a key range in the interval [lower,upper]
0360     virtual std::vector<Condition> get(DetElement              detector,
0361                                        Condition::itemkey_type lower,
0362                                        Condition::itemkey_type upper)  const;
0363       
0364     /// Interface to partially scan data content of the conditions mapping
0365     virtual void scan(DetElement                  detector,
0366                       Condition::itemkey_type     lower,
0367                       Condition::itemkey_type     upper,
0368                       const Condition::Processor& processor) const;
0369   };
0370 \end{unnumberedcode}
0371 Such $ConditionsMap$ implementations can easily be constructed using standard
0372 STL maps. The lookup key is constructed out of two elements:
0373 \begin{itemize}\itemcompact
0374 \item The detector element this condition belongs to and 
0375 \item an identifier of condition within this detector element.
0376 \end{itemize}
0377 An efficient implementation of a longword key would consist of the tuple:
0378 $$
0379 [ hash32(conditions\ name) , hash32(det-element\ path) ],
0380 $$
0381 which resembles to an ordered sequence of conditions according to 
0382 their detector element. A special implementation, which implements 
0383 this user interface is the $ConditionsSlice$ implemented in the 
0384 \DDC package (See section \ref{subsec:ddcond-conditions-store} for details).
0385 
0386 %=============================================================================
0387 \subsection{Common Conditions Tools}
0388 \label{subsec:ddcond-conditions-tools}
0389 %=============================================================================
0390 
0391 \noindent
0392 \begin{itemize}
0393 \item \bold{ConditionsPrinter} A tool to print conditions by scanning conditions for a 
0394         single $DetElement$ or the entire sub-tree. See 
0395         \detdesc{classdd4hep_1_1cond_1_1_conditions_printer.html}{DDCore/ConditionsPrinter.h} 
0396         for details).
0397 \item \bold{ConditionsProcessor} A wrapper to support condition functors implementing 
0398         the default callback:
0399         $$
0400         int\ operator()\ (Condition\ consition);
0401         $$
0402         The return value may be used to e.g. collect counters.
0403 \end{itemize}
0404 
0405 \newpage
0406 %=============================================================================
0407 \section{DDCond Conditions Store and Slices}
0408 \label{subsec:ddcond-conditions-store}
0409 %=============================================================================
0410 
0411 \noindent
0412 The $ConditionsMap$ interface allows tools to work with various conditions 
0413 data stores. \DDC provides an efficient implementation of such a store,
0414 which is described in the following chapters.
0415 
0416 %=============================================================================
0417 \subsection{Data Organization}
0418 \label{subsec:ddcond-internal-data-organization}
0419 %=============================================================================
0420 
0421 \noindent
0422 The basic assumption of the \DDC conditions store to optimize the access 
0423 and the management of conditions
0424 data can be very simply summarized: it is assumed, that groups of data items
0425 exist, which have a common interval of validity. In other words: given a 
0426 certain event, valid or invalid conditions can quickly be identified by 
0427 checking the so called "interval of validity" of the entire group with the
0428 time stamp of the event. This interval of validity defines the time span
0429 for which a given group of processing parameters is valid. It starts and 
0430 ends with a time stamp. The definition of a time stamp may be user defined 
0431 and not necessarily resemble to values in seconds or fractions thereof. 
0432 Time stamps could as well be formulated as an interval of luminosity sections,
0433 run numbers, fill numbers or entire years. 
0434 
0435 %=============================================================================
0436 \begin{figure}[t]
0437   \begin{center}\includegraphics[height=11cm] {DDCond-ConditionsStore}
0438     \caption{The graphical representation of the organisation of the
0439              conditions data in \DDH. }
0440     \label{fig:ddcond-data-organization}
0441   \end{center}
0442 \end{figure}
0443 
0444 \noindent
0445 Groups of parameters associated to certain intervals of validity can
0446 be very effectively managed if pooled together according to the 
0447 interval of validity. This of course assumes that each group contains
0448 a significant number of parameters. If each of these pools only contains
0449 one single value this would not be an efficient.
0450 
0451 \noindent
0452 This assumption is fundamental for this approach to be efficient. 
0453 If the data are not
0454 organized accordingly, the caching mechanism implemented in \DDC will 
0455 still work formally. However, by construction it cannot not work efficiently. 
0456 Resources both in CPU and memory would be wasted at run-time.
0457 The necessity to properly organize the conditions data becomes
0458 immediately evident in Figure~\ref{fig:ddcond-data-organization}:
0459 Users can organize data according to certain types, These types are
0460 independently managed and subdivided into pools. Each of these pools
0461 manages a set of conditions items sharing the same interval ov validity.
0462 
0463 \noindent
0464 The internal organization of the conditions data in \DDC is entirely 
0465 transparent to the user. The description here is contained for
0466 completeness and for the understanding of the limitations of the implemented
0467 approach. If different requirements or access patterns concerning the 
0468 access to conditions data arise, it should though be feasible to implement
0469 these fairly straight forward using a suited approach.
0470 
0471 %=============================================================================
0472 \subsection{Slice Configuration and Data Access}
0473 \label{subsec:ddcond-data-access}
0474 %=============================================================================
0475 
0476 \noindent
0477 As defined in section~\ref{subsec:ddcond-conditions-slices}, the conditions slice
0478 is the main entity to project conditions suitable to process a given particle
0479 collision (see \detdesc{classdd4hep_1_1cond_1_1_conditions_content.html}{DDCond/ConditionsContent.h}
0480 for details). Figure~\ref{fig:ddcond-slice-definition} shows the data content of a 
0481 conditions slice.
0482 As shown also in Figure~\ref{fig:ddcond-slice-usage}, there are 
0483 several steps to be performed before a conditions slice is ready to be used:
0484 \begin{enumerate}\itemcompact
0485 \item Create the conditions data slice.
0486 \item Setting up the data content of the slice by attaching an object of type
0487       $ConditionsContent$.
0488 \item Preparing the conditions data slice.
0489 \item Using the conditions data slice.
0490 \end{enumerate}
0491 
0492 %=============================================================================
0493 \begin{figure}[h]
0494   \begin{center}\includegraphics[width=12cm] {DDCond-ConditionsSlice}
0495     \caption{The data content of a $ConditionsSlice$ containing the 
0496              desired content ($ConditionsContent$), the pool to access the
0497              conditions data by key ($UserPool$) and optional containers
0498              showing the status of partial or unsuccessful load and prepare operations.}
0499     \label{fig:ddcond-slice-usage}
0500   \end{center}
0501 \end{figure}
0502 
0503 \noindent
0504 The $ConditionsContent$ 
0505 (see \detdesc{classdd4hep_1_1cond_1_1_conditions_slice.html}{DDCond/ConditionsSlice.h} 
0506 for details)  is a simple object, which contains load addresses
0507 to identify persistent conditions within the database/persistent schema 
0508 used and a set of dependency rules to compute the corresponding derived conditions.
0509 
0510 %=============================================================================
0511 \begin{figure}[t]
0512   \begin{center}\includegraphics[width=15cm] {DDCond-ConditionsAccess}
0513     \caption{The interaction of a user with the conditions
0514              data store using the $ConditionsSlice$ and the 
0515              $ConditionsManager$ interface to define the conditions content, 
0516              prepare the data and then access the loaded data from the slice
0517              projected according to the required interval of validity.}
0518     \label{fig:ddcond-slice-usage}
0519   \end{center}
0520 \end{figure}
0521 
0522 \noindent
0523 The $ConditionsSlice$ holds a consistent set of conditions valid for a given 
0524 interval of validity, which is the intersection of the intervals of 
0525 validity of all contained conditions. The has the following consequences 
0526 for the client when using a prepared $ConditionsSlice$:
0527 \begin{itemize}
0528 \item $ConditionsSlice$ objects are prepared by the client framework.
0529       Specific algorithms and other code fragments developed by physicist users
0530       should not deal with such activities. In multi-threaded applications
0531       the preparation of a $ConditionsSlice$ may be done in a separate thread.
0532 \item Once prepared, the slice nor the contained conditions may be altered.
0533       All contained conditions must be considered read-only.
0534 \item Since the slice is considered read-only, it can be used by multiple 
0535       clients simultaneously. In particular, multiple threads may share 
0536       the same slice.
0537 \item A $ConditionsSlice$ may only be re-used and prepared according to
0538       a different interval of validity once no other clients use it.
0539 \item At any point of time any number of $ConditionsSlice$ objects
0540       may be present in the client framework. There is no interference
0541       as long as the above mentioned requirements are fulfilled.
0542 \end{itemize}
0543 
0544 \noindent
0545 The fact that multiple instances of the conditions slices may be present 
0546 as well as the fact that the preparation of slices and their use is strictly
0547 separated makes then ideal for the usage within multi-threaded event 
0548 processing frameworks. As shown in 
0549 figure~\ref{fig:ddcond-multi-threaded-processing}, 
0550 the following use cases can easily be met:
0551 \begin{itemize}
0552 \item Mulitple threads may share the same slice while processing event data
0553       (thread 1-3) as long as the time stamp of the event data processed 
0554       by each thread is contained in the interval of validity of the
0555       slice.
0556 \item At the same time another thread may process event data with a different
0557       time stamp. The conditions for this event were prepared using another slice
0558       (thread 4-N).
0559 \end{itemize}
0560 
0561 %=============================================================================
0562 \begin{figure}[t]
0563   \begin{center}\includegraphics[width=15cm] {DDCond-ConditionsMT}
0564     \caption{MT.}
0565     \label{fig:ddcond-multi-threaded-processing}
0566   \end{center}
0567 \end{figure}
0568 \vspace{-0.5cm}
0569 
0570 
0571 %=============================================================================
0572 \subsection{Loading Conditions Data}
0573 \label{subsec:ddcond-data-loading}
0574 %=============================================================================
0575 
0576 \noindent
0577 The loading of conditions data is highly experiment specific. Different 
0578 access patterns and load implementations (single threaded, multi-threaded, locking etc.)
0579 make it close to impossible to implement any solution fitting all needs.
0580 For this reason the loading of conditions is deferred to an abstract implementation,
0581 which is invoked during the preparation phase of a conditions slice if the required
0582 data are not found in the conditions cache. This data loader interface
0583 (see \detdesc{classdd4hep_1_1cond_1_1_conditions_data_loader.html}{ConditionsDataLoader.h}
0584 for details), receives all relevant callbacks from the framework to resolve 
0585 missing conditions and pass the loaded objects to the framework for the management.
0586 The callback to be implemented by the framework developers are:
0587 \begin{unnumberedcode}
0588 /// Interface for a generic conditions loader
0589 /** 
0590  *  Common function for all loader.
0591  */
0592 class ConditionsDataLoader : public NamedObject, public PropertyConfigurable   {
0593   typedef Condition::key_type                                   key_type;
0594   typedef std::map<key_type,Condition>                          LoadedItems;
0595   typedef std::vector<std::pair<key_type,ConditionsLoadInfo*> > RequiredItems;
0596 public:
0597   ....
0598   /// Load a number of conditions items from the persistent medium according to the required IOV
0599   virtual size_t load_many(  const IOV&       req_validity,
0600                              RequiredItems&   work,
0601                              LoadedItems&     loaded,
0602                              IOV&             combined_validity) = 0;
0603 };
0604 \end{unnumberedcode}
0605 The arguments to the callback contain the necessary information to retrieve
0606 the requested items.
0607 
0608 \newpage
0609 %=============================================================================
0610 \section{Example Walkthrough}
0611 \label{sec:ddcond-example}
0612 %=============================================================================
0613 
0614 
0615 %=============================================================================
0616 \subsection{Example to Save Conditions to a ROOT File}
0617 \label{subsec:ddcond-example-saving}
0618 %=============================================================================
0619 
0620 \noindent
0621 To illustrate the usage of the \DDC package when saving conditions,
0622 an example is discussed here in detail.
0623 The full example is part of the conditions unit tests and can be found in the 
0624 \DDH examples.\\
0625 (See examples/Conditions/src/ConditionExample\_manual\_save.cpp).
0626 
0627 \noindent
0628 The examples uses conditions names and detector element names explicitly and
0629 hence requires a fixed detector description being loaded in memory.
0630 For simplicity we use here the Minitel example from the examples/ClientTests.
0631 However, the example is very generic and also the conditions are "generic",
0632 hence any geometry would work.
0633 
0634 
0635 \vspace{0.6cm}
0636 
0637 \noindent
0638 {\bf{Prerequisites:}} \\
0639 \\
0640 A valid compact geometry description to be loaded during the program startup.
0641 
0642 \vspace{0.6cm}
0643 
0644 \noindent
0645 {\bf{Plugin Invocation:}} \\
0646 \\
0647 A valid example conditions data file is required. Then use the default way
0648 to invoke plugins:\\
0649 \tw{\$ > geoPluginRun\ \textrm{-}destroy  \ \ \ \char`\\} \\
0650 \tw{\hspace{3cm} \textrm{-}plugin\ DD4hep\_ConditionExample\_manual\_save \ \ \ \char`\\} \\
0651 \tw{\hspace{3cm} \textrm{-}input\ file:\$\{DD4hep\_DIR\}/examples/AlignDet/compact/Telescope.xml
0652     \ \ \ \char`\\} \\
0653 \tw{\hspace{3cm} \textrm{-}conditions\ Conditions.root\ \textrm{-}runs\ 10}
0654 
0655 
0656 \vspace{0.6cm}
0657 
0658 \noindent
0659 {\bf{Example Code:}} \\
0660 
0661 \begin{code}
0662   int          num_run    = <number of condition loads>;
0663   const string conditions = <conditions file name>;
0664   const string geometry   = <geometry compact xml description";
0665 
0666   description.fromXML(geometry);
0667   description.apply("DD4hep_ConditionsManagerInstaller",0,(char**)0);
0668 
0669   ConditionsManager manager = ConditionsManager::from(description);
0670   manager["PoolType"]       = "DD4hep_ConditionsLinearPool";
0671   manager["UserPoolType"]   = "DD4hep_ConditionsMapUserPool";
0672   manager["UpdatePoolType"] = "DD4hep_ConditionsLinearUpdatePool";
0673   manager.initialize();
0674 
0675   shared_ptr<ConditionsContent> content(new ConditionsContent());
0676   shared_ptr<ConditionsSlice>   slice(new ConditionsSlice(manager,content));
0677 
0678   const IOVType*    iov_typ = manager.registerIOVType(0,"run").second;
0679   if ( 0 == iov_typ )
0680     except("ConditionsPrepare","++ Conditions IOV type registration failed!");
0681 
0682   Scanner(ConditionsKeys(*content,INFO),description.world());
0683   Scanner(ConditionsDependencyCreator(*content,DEBUG),description.world());
0684 
0685   // Have 10 run-slices [11,20] .... [91,100]
0686   for(int i=0; i<num_run; ++i)  {
0687     IOV iov(iov_typ, IOV::Key(1+i*10,(i+1)*10));
0688     ConditionsPool*   iov_pool = manager.registerIOV(*iov.iovType, iov.key());
0689     // Create conditions with all deltas. Use a generic creator
0690     Scanner(ConditionsCreator(*slice, *iov_pool, INFO),description.world(),0,true);
0691   }
0692 
0693 \end{code}
0694 
0695 \noindent
0696 {\bf{Explanation:}} \\
0697 \begin{tabular} {l||p{0cm}}
0698 \docline{Line}{}
0699 \docline{1-3}{Definition of processing parameters.}
0700 \docline{5}{Load the detector description using the compact notation.}
0701 \docline{6}{Install the conditions manager implementation using the plugin mechanism.}
0702 \docline{8}{Access conditions manager instance from the Detector interface.}
0703 \docline{9-11}{Configure the properties of the conditions manager.}
0704 \docline{12}{Initialize the conditions manager instance.}
0705 \docline{14-15}{Create an empty $ConditionsSlice$ instance the container with 
0706                 the desired conditions content.}
0707 \docline{17}{Register IOV type the Conditions Manager. The IOV types are part of the conditions
0708             persistency mechanism. They may not change with time and have to be defined 
0709             by the experiment once and for all!}
0710 \docline{18-19}{This is example specific and only a shortcut to fill the 
0711              required conditions content and the derivation rules.\\
0712              In real life this would be intrinsic to the experiment's 
0713              data processing framework.}
0714 \docline{18}{Populate the $ConditionsContent$ instance with the addresses (keys) 
0715              of the conditions required:
0716              We scan the $DetElement$ hierarchy and add a couple of conditions
0717              for each $DetElement$}
0718 \docline{19}{Add for each $DetElement$ 3 derived conditions,
0719              which all depend on the persistent condition derived\_data.\\
0720              In the real world this would be very specific derived actions.}
0721 
0722 \end{tabular}
0723 
0724 \newpage
0725 
0726 
0727 %=============================================================================
0728 \subsection{Example to Load and Prepare Conditions(Slices)}
0729 \label{subsec:ddcond-example-loading}
0730 %=============================================================================
0731 
0732 \noindent
0733 To illustrate the usage of the \DDC package when loading conditions,
0734 an example is discussed here in detail.
0735 The full example is part of the conditions unit tests and can be found in the 
0736 \DDH examples.\\
0737 (See examples/Conditions/src/ConditionExample\_manual\_load.cpp).
0738 
0739 \noindent
0740 The examples uses conditions names and detector element names explicitly and
0741 hence requires a fixed detector description being loaded in memory.
0742 For simplicity we use here the Minitel example from the examples/ClientTests.
0743 However, the example is very generic and also the conditions are "generic",
0744 hence any geometry would work.
0745 
0746 \vspace{0.6cm}
0747 
0748 \noindent
0749 {\bf{Prerequisites:}} \\
0750 \\
0751 A valid example conditions data file is required, since in this example we 
0752 load the conditions and inject them to the store from an already existing root file.
0753 To obtain such a file for a given geometry, execute the example plugin:\\
0754 \tw{\$ > geoPluginRun\ \textrm{-}destroy  \ \ \ \char`\\} \\
0755 \tw{\hspace{3cm} \textrm{-}plugin\ DD4hep\_ConditionExample\_manual\_save \ \ \ \char`\\} \\
0756 \tw{\hspace{3cm} \textrm{-}input\ file:\$\{DD4hep\_DIR\}/examples/AlignDet/compact/Telescope.xml
0757     \ \ \ \char`\\} \\
0758 \tw{\hspace{3cm} \textrm{-}conditions Conditions.root\ \textrm{-}runs\ 10}
0759 
0760 \vspace{0.6cm}
0761 
0762 \noindent
0763 {\bf{Plugin Invocation:}} \\
0764 \\
0765 A valid example conditions data file is required. Then use the default way
0766 to invoke plugins:\\
0767 \tw{\$ > geoPluginRun\ \textrm{-}destroy  \ \ \ \char`\\} \\
0768 \tw{\hspace{3cm} \textrm{-}plugin\ DD4hep\_ConditionExample\_manual\_load \ \ \ \char`\\} \\
0769 \tw{\hspace{3cm} \textrm{-}input\ file:\$\{DD4hep\_DIR\}/examples/AlignDet/compact/Telescope.xml
0770     \ \ \ \char`\\} \\
0771 \tw{\hspace{3cm} \textrm{-}conditions\ Conditions.root\ \textrm{-}runs\ 10}
0772 
0773 
0774 \vspace{0.6cm}
0775 
0776 \noindent
0777 {\bf{Example Code:}} \\
0778 
0779 \begin{code}
0780   int          num_run    = <number of condition loads>;
0781   const string conditions = <conditions file name>;
0782   const string geometry   = <geometry compact xml description";
0783 
0784   description.fromXML(geometry);
0785   description.apply("DD4hep_ConditionsManagerInstaller",0,(char**)0);
0786 
0787   ConditionsManager manager = ConditionsManager::from(description);
0788   manager["PoolType"]       = "DD4hep_ConditionsLinearPool";
0789   manager["UserPoolType"]   = "DD4hep_ConditionsMapUserPool";
0790   manager["UpdatePoolType"] = "DD4hep_ConditionsLinearUpdatePool";
0791   manager["LoaderType"]     = "root";
0792   manager.initialize();
0793 
0794   shared_ptr<ConditionsContent> content(new ConditionsContent());
0795   shared_ptr<ConditionsSlice>   slice(new ConditionsSlice(manager,content));
0796 
0797   Scanner(ConditionsKeys(*content,INFO),description.world());
0798   Scanner(ConditionsDependencyCreator(*content,DEBUG),description.world());
0799 
0800   const IOVType* iov_typ = manager.iovType("run");
0801   for ( int irun=0; irun < num_runs; ++irun )  {
0802     IOV iov(iov_typ,irun*10+5);
0803     ConditionsManager::Result r = manager.prepare(iov,*slice);
0804     if ( r.missing != 0 )  {
0805        except("Example", 
0806               "Conditions prepare step for IOV %s FAILED. There are %ld missing conditions.",
0807               r.missing, iov.str().c_str());
0808     }
0809     Scanner(ConditionsPrinter(slice.get(),"Example"),description.world());
0810   }
0811 \end{code}
0812 
0813 \noindent
0814 {\bf{Explanation:}} \\
0815 \begin{tabular} {l||p{0cm}}
0816 \docline{Line}{}
0817 \docline{1-3}{Definition of processing parameters.}
0818 \docline{5}{Load the detector description using the compact notation.}
0819 \docline{6}{Install the conditions manager implementation using the plugin mechanism.}
0820 \docline{8}{Access conditions manager instance from the Detector interface.}
0821 \docline{9-12}{Configure the properties of the conditions manager.}
0822 \docline{13}{Initialize the conditions manager instance.}
0823 \docline{15-16}{Create an empty $ConditionsSlice$ instance the container with 
0824                 the desired conditions content.}
0825 \docline{18-19}{This is example specific and only a shortcut to fill the 
0826              required conditions content and the derivation rules.\\
0827              In real life this would be intrinsic to the experiment's 
0828              data processing framework.}
0829 \docline{18}{Populate the $ConditionsContent$ instance with the addresses (keys) 
0830              of the conditions required:
0831              We scan the $DetElement$ hierarchy and add a couple of conditions
0832              for each $DetElement$}
0833 \docline{19}{Add for each $DetElement$ 3 derived conditions,
0834              which all depend on the persistent condition derived\_data.\\
0835              In the real world this would be very specific derived actions.}
0836 
0837 \docline{22-31}{Emulate a pseudo event loop: Our conditions are of type "run".}
0838 \docline{23}{This is the IOV we want to use for this "processing step". 
0839             The conditions filled into the slice during the prepare step will 
0840             satisfy this IOV requirement.}
0841 \docline{24}{Conditions prepare step. Select the proper set of conditions 
0842             from the store (or load them if needed). Attach the selected 
0843             conditions to the user pool.}
0844 \docline{25-29}{Check the result of the prepare step. If anything would be missing,
0845             the parameter r.missing of the return code would be non-zero.}
0846 \docline{30}{Emulate data processing algorithms: Here we only scan the 
0847             $DetElement$ tree and print all conditions.\\
0848             We know what we expect since we defined the content the same way!
0849             In the printer we can access the conditions directly from the slice,
0850             since the slice implements the $ConditionsMap$ interface.}
0851 \end{tabular}
0852 
0853 \newpage
0854 %=============================================================================
0855 \begin{thebibliography}{9}
0856 \bibitem{bib:dd4hep-manual}  M.Frank, \DDH manual.
0857 \bibitem{bib:ddalign-manual}  M.Frank, \DDA manual.
0858 \end{thebibliography}
0859 %=============================================================================
0860 \end{document}