File indexing completed on 2026-04-09 07:58:32
0001 <!DOCTYPE HTML>
0002
0003
0004
0005
0006
0007 <html>
0008 <head>
0009 <title>iDDS</title>
0010 <meta charset="utf-8" />
0011 <meta name="viewport" content="width=device-width, initial-scale=1" />
0012 <link rel="stylesheet" href="assets/css/main.css" />
0013 </head>
0014 <body>
0015
0016
0017 <header id="header" class="alt">
0018 <div class="logo"><a href="index.html">iDDS <span>intelligent Data Delivery Serivce</span></a></div>
0019 <a href="#menu">Menu</a>
0020 </header>
0021
0022
0023 <nav id="menu">
0024 <ul class="links">
0025 <li><a href="index.html">Home</a></li>
0026 <li><a href="https://github.com/HSF/iDDS">Github Codes</a></li>
0027 <li><a href="https://idds.readthedocs.io">Documents</a></li>
0028
0029
0030 </ul>
0031 </nav>
0032
0033
0034 <section class="banner full">
0035 <article>
0036
0037 <img src="images/idds_main.jpg" alt="" />
0038 <div class="inner">
0039 <header>
0040 <p><b>Intelligent</b></p>
0041 <p><b>Transformation</b></p>
0042 <p><b>Delivery</b></p>
0043 <p><b>Orchestration</b></p>
0044 <h2>iDDS</h2>
0045 </header>
0046 </div>
0047 </article>
0048 <article>
0049
0050 <img src="images/idds_intelligent.jpg" alt="" />
0051 <div class="inner">
0052 <header>
0053 <p><b>Intelligent optimize the workflow</b></p>
0054 <h2>Intelligent</h2>
0055 </header>
0056 </div>
0057 </article>
0058 <article>
0059
0060 <img src="images/idds_transform.jpg" alt="" />
0061 <div class="inner">
0062 <header>
0063 <p><b>Fine grained data transformation</b></p>
0064 <p><b>Remote data transformation/reduction</b></p>
0065 <p><b>On-demand production of analysis format data transformation</b></p>
0066 <h2>Transformation</h2>
0067 </header>
0068 </div>
0069 </article>
0070 <article>
0071
0072 <img src="images/idds_delivery.jpg" alt="" />
0073 <div class="inner">
0074 <header>
0075 <p><b>Deliver only when needed</b></p>
0076 <p><b>Fine grained tape carousel</b></p>
0077 <h2>Delivery</h2>
0078 </header>
0079 </div>
0080 </article>
0081 <article>
0082
0083 <img src="images/idds_orchestration.jpg" alt="" />
0084 <div class="inner">
0085 <header>
0086 <p><b>Orchestration of <a href="https://bigpanda.cern.ch">WFMS</a> and <a href="https://rucio.cern.ch">DDMS</a> with generalized workflows</b></p>
0087 <h2>Orchestration</h2>
0088 </header>
0089 </div>
0090 </article>
0091
0092
0093
0094
0095
0096
0097
0098
0099
0100
0101
0102 </section>
0103
0104
0105 <section id="zero" class="wrapper style3">
0106 <div class="inner">
0107 <header class="align-center">
0108 <p><b>A general service to extract and abstract common functions of <a href="https://bigpanda.cern.ch">WFMS</a> and <a href="https://rucio.cern.ch">DDMS</a></b></p>
0109 <p><b>Flexibility to experiment agnostic</b></p>
0110 <p><b>Flexibility to support different use-cases and backends with plugin structure</b></p>
0111 <h2>Intelligent to transform and deliver needed data to consumers </h2>
0112 </header>
0113 </div>
0114 </section>
0115
0116
0117 <section id="one" class="wrapper style1">
0118 <div class="inner">
0119 <header class="align-center">
0120 <p class="special"><b>Components structure and Layers Architecture</b></p>
0121 <h2>Architecture</h2>
0122 </header>
0123
0124 <div class="grid-style">
0125
0126 <div>
0127 <div class="box">
0128 <div class="image fit">
0129
0130 <img src="images/idds_components.png" alt="" />
0131 </div>
0132 <div class="content" color="inherit" background-color="inherit">
0133 <header class="align-center">
0134 <p>Main Components</p>
0135 <h2>Components</h2>
0136 </header>
0137 <p>
0138 <b>Transformer</b> To transform input data from one collection to another collection. With plugin structure to support different transform type and different backends.<br>
0139 <b>Transporter</b> To manage collections with DDM backend.<br>
0140 <b>Conductor</b> To notify/schedule consumers to consume new transformed data in a fine-grained granularity.
0141 </p>
0142
0143 <footer class="align-center">
0144 <a href="#" class="button alt">Learn More</a>
0145 </footer>
0146 </div>
0147 </div>
0148 </div>
0149
0150 <div>
0151 <div class="box">
0152 <div class="image fit">
0153
0154 <img src="images/idds_architecture.png" alt="" />
0155 </div>
0156 <div class="content">
0157 <header class="align-center">
0158 <p>Layer and Plugin Architecture</p>
0159 <h2>Layers and Plugins</h2>
0160 </header>
0161 <p>
0162 <b>Abstract Layers</b> Hide the complexity of different logics to implement general API<br>
0163 <b>Plugins</b> To support different transforms and backends<br>
0164 <b>Restful interface</b> Interface to communicate with clients<br>
0165 </p>
0166 <footer class="align-center">
0167 <a href="#" class="button alt">Learn More</a>
0168 </footer>
0169 </div>
0170 </div>
0171 </div>
0172
0173 </div>
0174 </div>
0175 </section>
0176
0177
0178 <section id="two" class="wrapper style3">
0179 <div class="inner">
0180 <header class="align-center">
0181 <p><b>Experiment agnostic based on generation</b></p>
0182 <p><b>Abstract for ochestration</b></p>
0183 <p><b>Maintainability and extensibility with plugins</b></p>
0184 <h2>Features</h2>
0185 </header>
0186 </div>
0187 </section>
0188
0189
0190 <section id="three" class="wrapper style2">
0191 <div class="inner">
0192 <header class="align-center">
0193 <p class="special"><b>Applications that iDDS would like to serve</b></p>
0194 <h2>Potential Applications</h2>
0195 </header>
0196
0197 <ul class="features">
0198 <li class="icon fa-lightbulb-o">
0199 <h3>Event Streaming Service</h3>
0200 <p>Split files into fine grained events and stream the events as inputs for EventService.</p>
0201 </li>
0202 <li class="icon fa-cloud">
0203 <h3>Fine grained tape carousel</h3>
0204 <p>Aggregate tape stage-in requests and immediate processing of partially staged data.</p>
0205 </li>
0206 <li class="icon fa-cogs">
0207 <h3>On-demand DAOD production</h3>
0208 <p>Passive creation of analysis format data rather than proactively keeing them on DISK, reduction of expensive disk usage while increaing of cheap CPU usage.</p>
0209 </li>
0210 <li class="icon fa-lock">
0211 <h3>Fine grained remoted data transformation/reduction</h3>
0212 <p>Remotely transform/reduce data and then deliver only needed data to consumers.</p>
0213 </li>
0214
0215
0216
0217
0218
0219
0220
0221
0222
0223
0224
0225
0226
0227
0228
0229
0230
0231
0232 </ul>
0233
0234
0235
0236
0237
0238
0239
0240
0241
0242
0243
0244
0245
0246
0247
0248
0249
0250
0251
0252
0253
0254
0255
0256
0257 </div>
0258 </section>
0259
0260
0261
0262 <footer id="footer">
0263 <div class="container">
0264 <ul class="icons">
0265 <li><a href="#" class="icon fa-twitter"><span class="label">Twitter</span></a></li>
0266 <li><a href="#" class="icon fa-facebook"><span class="label">Facebook</span></a></li>
0267 <li><a href="#" class="icon fa-instagram"><span class="label">Instagram</span></a></li>
0268 <li><a href="#" class="icon fa-envelope-o"><span class="label">Email</span></a></li>
0269 </ul>
0270 </div>
0271 <div class="copyright">
0272 © iDDS. All rights reserved.
0273 </div>
0274 </footer>
0275
0276
0277 <script src="assets/js/jquery.min.js"></script>
0278 <script src="assets/js/jquery.scrollex.min.js"></script>
0279 <script src="assets/js/skel.min.js"></script>
0280 <script src="assets/js/util.js"></script>
0281 <script src="assets/js/main.js"></script>
0282
0283 </body>
0284 </html>
