docs/howto/streaming.md

0001 # Stream data to and from JANA <!-- {docsify-ignore-all} -->
0002
0003
0004 ## Messages and Events
0005
0006 1. The first question to ask is: What is the relationship between messages and events? Remember, a message is just
0007    a packet of data sent over the wire, whereas an event is JANA's main unit of independent computation, corresponding
0008    to all data associated with one physics interaction. The answer will depend on:
0009
0010    - What systems already exist upstream, and how difficult they are to change
0011    - The expected size of each event
0012    - Whether event building is handled upstream or within JANA
0013
0014    If events are large enough (>0.5MB), the cleanest thing to do is to
0015    establish a one-to-one relationship between messages and events. JANA provides
0016    [JStreamingEventSource](refcpp/class_j_streaming_event_source.html)
0017    to make this convenient.
0018
0019    If events are very small, you probably want many events in one message. A corresponding helper class does not
0020    exist yet, but would be a straightforward adaptation of the above.
0021
0022    If upstream doesn't do any event building (e.g. it is reading out ADC samples over a fixed time window) you
0023    probably want to have JANA determine physically meaningful event boundaries, maybe even incorporating a software
0024    L2 trigger. This is considerably more complicated, and is discussed in [the event building how-to](./other-howtos.md)
0025    instead.
0026
0027    For the remainder of this how-to we assume that messages and events are one-to-one.
0028
0029
0030 ## Transport
0031
0032 2. The second question to ask is: What transport should be used?
0033
0034     JANA makes it so that the message format and transport can be varied independently. The transport wrapper need only
0035     implement the [JTransport](refcpp/struct_j_transport.html) interface, which is essentially just:
0036
0037     ```cpp
0038         enum class Result {SUCCESS, TRYAGAIN};
0039
0040         virtual void initialize();
0041         virtual Result send(const JMessage& src_msg);
0042         virtual Result receive(JMessage& dest_msg);
0043     ```
0044
0045     The key detail is that both `send` and `receive` should block until data has finished transferring to/from the `JMessage`
0046     buffer so that the buffer may be accessed by the caller with no additional synchronization. If there are no pending
0047      messages, `receive` should return `TRYAGAIN` immediately so as not to block the event source. In contrast,
0048      `send` must block until it succeeds, as otherwise there will be data loss.
0049
0050     An implementation already exists for ZeroMQ. See `examples/JExample7/ZmqTransport.h`
0051
0052 ## Message format
0053
0054 3. The final and most important question to ask is: What is the message format?
0055
0056     Message formats each get their own class, which must inherit from the JMessage and JEventMessage interfaces.
0057
0058
0059