![]() |
|
|||
File indexing completed on 2025-04-18 09:16:01
0001 /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 0002 * Copyright by The HDF Group. * 0003 * All rights reserved. * 0004 * * 0005 * This file is part of HDF5. The full HDF5 copyright notice, including * 0006 * terms governing use, modification, and redistribution, is contained in * 0007 * the COPYING file, which can be found at the root of the source code * 0008 * distribution tree, or in https://www.hdfgroup.org/licenses. * 0009 * If you do not have access to either file, you may request a copy from * 0010 * help@hdfgroup.org. * 0011 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ 0012 0013 /*------------------------------------------------------------------------- 0014 * 0015 * Created: H5ACpublic.h 0016 * 0017 * Purpose: Public include file for cache functions 0018 * 0019 *------------------------------------------------------------------------- 0020 */ 0021 #ifndef H5ACpublic_H 0022 #define H5ACpublic_H 0023 0024 #include "H5public.h" /* Generic Functions */ 0025 #include "H5Cpublic.h" /* Cache */ 0026 0027 /**************************************************************************** 0028 * 0029 * structure H5AC_cache_config_t 0030 * 0031 * H5AC_cache_config_t is a public structure intended for use in public APIs. 0032 * At least in its initial incarnation, it is basically a copy of struct 0033 * H5C_auto_size_ctl_t, minus the report_fcn field, and plus the 0034 * dirty_bytes_threshold field. 0035 * 0036 * The report_fcn field is omitted, as including it would require us to 0037 * make H5C_t structure public. 0038 * 0039 * The dirty_bytes_threshold field does not appear in H5C_auto_size_ctl_t, 0040 * as synchronization between caches on different processes is handled at 0041 * the H5AC level, not at the level of H5C. Note however that there is 0042 * considerable interaction between this value and the other fields in this 0043 * structure. 0044 * 0045 * Similarly, the open_trace_file, close_trace_file, and trace_file_name 0046 * fields do not appear in H5C_auto_size_ctl_t, as most trace file 0047 * issues are handled at the H5AC level. The one exception is storage of 0048 * the pointer to the trace file, which is handled by H5C. 0049 * 0050 * The structure is in H5ACpublic.h as we may wish to allow different 0051 * configuration options for metadata and raw data caches. 0052 * 0053 * The fields of the structure are discussed individually below: 0054 * 0055 * version: Integer field containing the version number of this version 0056 * of the H5AC_cache_config_t structure. Any instance of 0057 * H5AC_cache_config_t passed to the cache must have a known 0058 * version number, or an error will be flagged. 0059 * 0060 * rpt_fcn_enabled: Boolean field used to enable and disable the default 0061 * reporting function. This function is invoked every time the 0062 * automatic cache resize code is run, and reports on its activities. 0063 * 0064 * This is a debugging function, and should normally be turned off. 0065 * 0066 * open_trace_file: Boolean field indicating whether the trace_file_name 0067 * field should be used to open a trace file for the cache. 0068 * 0069 * *** DEPRECATED *** Use H5Fstart/stop logging functions instead 0070 * 0071 * The trace file is a debugging feature that allow the capture of 0072 * top level metadata cache requests for purposes of debugging and/or 0073 * optimization. This field should normally be set to false, as 0074 * trace file collection imposes considerable overhead. 0075 * 0076 * This field should only be set to true when the trace_file_name 0077 * contains the full path of the desired trace file, and either 0078 * there is no open trace file on the cache, or the close_trace_file 0079 * field is also true. 0080 * 0081 * close_trace_file: Boolean field indicating whether the current trace 0082 * file (if any) should be closed. 0083 * 0084 * *** DEPRECATED *** Use H5Fstart/stop logging functions instead 0085 * 0086 * See the above comments on the open_trace_file field. This field 0087 * should be set to false unless there is an open trace file on the 0088 * cache that you wish to close. 0089 * 0090 * trace_file_name: Full path of the trace file to be opened if the 0091 * open_trace_file field is true. 0092 * 0093 * *** DEPRECATED *** Use H5Fstart/stop logging functions instead 0094 * 0095 * In the parallel case, an ascii representation of the mpi rank of 0096 * the process will be appended to the file name to yield a unique 0097 * trace file name for each process. 0098 * 0099 * The length of the path must not exceed H5AC__MAX_TRACE_FILE_NAME_LEN 0100 * characters. 0101 * 0102 * evictions_enabled: Boolean field used to either report the current 0103 * evictions enabled status of the cache, or to set the cache's 0104 * evictions enabled status. 0105 * 0106 * In general, the metadata cache should always be allowed to 0107 * evict entries. However, in some cases it is advantageous to 0108 * disable evictions briefly, and thereby postpone metadata 0109 * writes. However, this must be done with care, as the cache 0110 * can grow quickly. If you do this, re-enable evictions as 0111 * soon as possible and monitor cache size. 0112 * 0113 * At present, evictions can only be disabled if automatic 0114 * cache resizing is also disabled (that is, ( incr_mode == 0115 * H5C_incr__off ) && ( decr_mode == H5C_decr__off )). There 0116 * is no logical reason why this should be so, but it simplifies 0117 * implementation and testing, and I can't think of any reason 0118 * why it would be desirable. If you can think of one, I'll 0119 * revisit the issue. 0120 * 0121 * set_initial_size: Boolean flag indicating whether the size of the 0122 * initial size of the cache is to be set to the value given in 0123 * the initial_size field. If set_initial_size is false, the 0124 * initial_size field is ignored. 0125 * 0126 * initial_size: If enabled, this field contain the size the cache is 0127 * to be set to upon receipt of this structure. Needless to say, 0128 * initial_size must lie in the closed interval [min_size, max_size]. 0129 * 0130 * min_clean_fraction: double in the range 0 to 1 indicating the fraction 0131 * of the cache that is to be kept clean. This field is only used 0132 * in parallel mode. Typical values are 0.1 to 0.5. 0133 * 0134 * max_size: Maximum size to which the cache can be adjusted. The 0135 * supplied value must fall in the closed interval 0136 * [MIN_MAX_CACHE_SIZE, MAX_MAX_CACHE_SIZE]. Also, max_size must 0137 * be greater than or equal to min_size. 0138 * 0139 * min_size: Minimum size to which the cache can be adjusted. The 0140 * supplied value must fall in the closed interval 0141 * [H5C__MIN_MAX_CACHE_SIZE, H5C__MAX_MAX_CACHE_SIZE]. Also, min_size 0142 * must be less than or equal to max_size. 0143 * 0144 * epoch_length: Number of accesses on the cache over which to collect 0145 * hit rate stats before running the automatic cache resize code, 0146 * if it is enabled. 0147 * 0148 * At the end of an epoch, we discard prior hit rate data and start 0149 * collecting afresh. The epoch_length must lie in the closed 0150 * interval [H5C__MIN_AR_EPOCH_LENGTH, H5C__MAX_AR_EPOCH_LENGTH]. 0151 * 0152 * 0153 * Cache size increase control fields: 0154 * 0155 * incr_mode: Instance of the H5C_cache_incr_mode enumerated type whose 0156 * value indicates how we determine whether the cache size should be 0157 * increased. At present there are two possible values: 0158 * 0159 * H5C_incr__off: Don't attempt to increase the size of the cache 0160 * automatically. 0161 * 0162 * When this increment mode is selected, the remaining fields 0163 * in the cache size increase section ar ignored. 0164 * 0165 * H5C_incr__threshold: Attempt to increase the size of the cache 0166 * whenever the average hit rate over the last epoch drops 0167 * below the value supplied in the lower_hr_threshold 0168 * field. 0169 * 0170 * Note that this attempt will fail if the cache is already 0171 * at its maximum size, or if the cache is not already using 0172 * all available space. 0173 * 0174 * Note that you must set decr_mode to H5C_incr__off if you 0175 * disable metadata cache entry evictions. 0176 * 0177 * lower_hr_threshold: Lower hit rate threshold. If the increment mode 0178 * (incr_mode) is H5C_incr__threshold and the hit rate drops below the 0179 * value supplied in this field in an epoch, increment the cache size by 0180 * size_increment. Note that cache size may not be incremented above 0181 * max_size, and that the increment may be further restricted by the 0182 * max_increment field if it is enabled. 0183 * 0184 * When enabled, this field must contain a value in the range [0.0, 1.0]. 0185 * Depending on the incr_mode selected, it may also have to be less than 0186 * upper_hr_threshold. 0187 * 0188 * increment: Double containing the multiplier used to derive the new 0189 * cache size from the old if a cache size increment is triggered. 0190 * The increment must be greater than 1.0, and should not exceed 2.0. 0191 * 0192 * The new cache size is obtained my multiplying the current max cache 0193 * size by the increment, and then clamping to max_size and to stay 0194 * within the max_increment as necessary. 0195 * 0196 * apply_max_increment: Boolean flag indicating whether the max_increment 0197 * field should be used to limit the maximum cache size increment. 0198 * 0199 * max_increment: If enabled by the apply_max_increment field described 0200 * above, this field contains the maximum number of bytes by which the 0201 * cache size can be increased in a single re-size. 0202 * 0203 * flash_incr_mode: Instance of the H5C_cache_flash_incr_mode enumerated 0204 * type whose value indicates whether and by which algorithm we should 0205 * make flash increases in the size of the cache to accommodate insertion 0206 * of large entries and large increases in the size of a single entry. 0207 * 0208 * The addition of the flash increment mode was occasioned by performance 0209 * problems that appear when a local heap is increased to a size in excess 0210 * of the current cache size. While the existing re-size code dealt with 0211 * this eventually, performance was very bad for the remainder of the 0212 * epoch. 0213 * 0214 * At present, there are two possible values for the flash_incr_mode: 0215 * 0216 * H5C_flash_incr__off: Don't perform flash increases in the size of 0217 * the cache. 0218 * 0219 * H5C_flash_incr__add_space: Let x be either the size of a newly 0220 * newly inserted entry, or the number of bytes by which the 0221 * size of an existing entry has been increased. 0222 * 0223 * If 0224 * x > flash_threshold * current max cache size, 0225 * 0226 * increase the current maximum cache size by x * flash_multiple 0227 * less any free space in the cache, and star a new epoch. For 0228 * now at least, pay no attention to the maximum increment. 0229 * 0230 * In both of the above cases, the flash increment pays no attention to 0231 * the maximum increment (at least in this first incarnation), but DOES 0232 * stay within max_size. 0233 * 0234 * With a little thought, it should be obvious that the above flash 0235 * cache size increase algorithm is not sufficient for all circumstances 0236 * -- for example, suppose the user round robins through 0237 * (1/flash_threshold) +1 groups, adding one data set to each on each 0238 * pass. Then all will increase in size at about the same time, requiring 0239 * the max cache size to at least double to maintain acceptable 0240 * performance, however the above flash increment algorithm will not be 0241 * triggered. 0242 * 0243 * Hopefully, the add space algorithms detailed above will be sufficient 0244 * for the performance problems encountered to date. However, we should 0245 * expect to revisit the issue. 0246 * 0247 * flash_multiple: Double containing the multiple described above in the 0248 * H5C_flash_incr__add_space section of the discussion of the 0249 * flash_incr_mode section. This field is ignored unless flash_incr_mode 0250 * is H5C_flash_incr__add_space. 0251 * 0252 * flash_threshold: Double containing the factor by which current max cache 0253 * size is multiplied to obtain the size threshold for the add_space flash 0254 * increment algorithm. The field is ignored unless flash_incr_mode is 0255 * H5C_flash_incr__add_space. 0256 * 0257 * 0258 * Cache size decrease control fields: 0259 * 0260 * decr_mode: Instance of the H5C_cache_decr_mode enumerated type whose 0261 * value indicates how we determine whether the cache size should be 0262 * decreased. At present there are four possibilities. 0263 * 0264 * H5C_decr__off: Don't attempt to decrease the size of the cache 0265 * automatically. 0266 * 0267 * When this increment mode is selected, the remaining fields 0268 * in the cache size decrease section are ignored. 0269 * 0270 * H5C_decr__threshold: Attempt to decrease the size of the cache 0271 * whenever the average hit rate over the last epoch rises 0272 * above the value supplied in the upper_hr_threshold 0273 * field. 0274 * 0275 * H5C_decr__age_out: At the end of each epoch, search the cache for 0276 * entries that have not been accessed for at least the number 0277 * of epochs specified in the epochs_before_eviction field, and 0278 * evict these entries. Conceptually, the maximum cache size 0279 * is then decreased to match the new actual cache size. However, 0280 * this reduction may be modified by the min_size, the 0281 * max_decrement, and/or the empty_reserve. 0282 * 0283 * H5C_decr__age_out_with_threshold: Same as age_out, but we only 0284 * attempt to reduce the cache size when the hit rate observed 0285 * over the last epoch exceeds the value provided in the 0286 * upper_hr_threshold field. 0287 * 0288 * Note that you must set decr_mode to H5C_decr__off if you 0289 * disable metadata cache entry evictions. 0290 * 0291 * upper_hr_threshold: Upper hit rate threshold. The use of this field 0292 * varies according to the current decr_mode: 0293 * 0294 * H5C_decr__off or H5C_decr__age_out: The value of this field is 0295 * ignored. 0296 * 0297 * H5C_decr__threshold: If the hit rate exceeds this threshold in any 0298 * epoch, attempt to decrement the cache size by size_decrement. 0299 * 0300 * Note that cache size may not be decremented below min_size. 0301 * 0302 * Note also that if the upper_threshold is 1.0, the cache size 0303 * will never be reduced. 0304 * 0305 * H5C_decr__age_out_with_threshold: If the hit rate exceeds this 0306 * threshold in any epoch, attempt to reduce the cache size 0307 * by evicting entries that have not been accessed for more 0308 * than the specified number of epochs. 0309 * 0310 * decrement: This field is only used when the decr_mode is 0311 * H5C_decr__threshold. 0312 * 0313 * The field is a double containing the multiplier used to derive the 0314 * new cache size from the old if a cache size decrement is triggered. 0315 * The decrement must be in the range 0.0 (in which case the cache will 0316 * try to contract to its minimum size) to 1.0 (in which case the 0317 * cache will never shrink). 0318 * 0319 * apply_max_decrement: Boolean flag used to determine whether decrements 0320 * in cache size are to be limited by the max_decrement field. 0321 * 0322 * max_decrement: Maximum number of bytes by which the cache size can be 0323 * decreased in a single re-size. Note that decrements may also be 0324 * restricted by the min_size of the cache, and (in age out modes) by 0325 * the empty_reserve field. 0326 * 0327 * epochs_before_eviction: Integer field used in H5C_decr__age_out and 0328 * H5C_decr__age_out_with_threshold decrement modes. 0329 * 0330 * This field contains the number of epochs an entry must remain 0331 * unaccessed before it is evicted in an attempt to reduce the 0332 * cache size. If applicable, this field must lie in the range 0333 * [1, H5C__MAX_EPOCH_MARKERS]. 0334 * 0335 * apply_empty_reserve: Boolean field controlling whether the empty_reserve 0336 * field is to be used in computing the new cache size when the 0337 * decr_mode is H5C_decr__age_out or H5C_decr__age_out_with_threshold. 0338 * 0339 * empty_reserve: To avoid a constant racheting down of cache size by small 0340 * amounts in the H5C_decr__age_out and H5C_decr__age_out_with_threshold 0341 * modes, this field allows one to require that any cache size 0342 * reductions leave the specified fraction of unused space in the cache. 0343 * 0344 * The value of this field must be in the range [0.0, 1.0]. I would 0345 * expect typical values to be in the range of 0.01 to 0.1. 0346 * 0347 * 0348 * Parallel Configuration Fields: 0349 * 0350 * In PHDF5, all operations that modify metadata must be executed collectively. 0351 * 0352 * We used to think that this was enough to ensure consistency across the 0353 * metadata caches, but since we allow processes to read metadata individually, 0354 * the order of dirty entries in the LRU list can vary across processes, 0355 * which can result in inconsistencies between the caches. 0356 * 0357 * PHDF5 uses several strategies to prevent such inconsistencies in metadata, 0358 * all of which use the fact that the same stream of dirty metadata is seen 0359 * by all processes for purposes of synchronization. This is done by 0360 * having each process count the number of bytes of dirty metadata generated, 0361 * and then running a "sync point" whenever this count exceeds a user 0362 * specified threshold (see dirty_bytes_threshold below). 0363 * 0364 * The current metadata write strategy is indicated by the 0365 * metadata_write_strategy field. The possible values of this field, along 0366 * with the associated metadata write strategies are discussed below. 0367 * 0368 * dirty_bytes_threshold: Threshold of dirty byte creation used to 0369 * synchronize updates between caches. (See above for outline and 0370 * motivation.) 0371 * 0372 * This value MUST be consistent across all processes accessing the 0373 * file. This field is ignored unless HDF5 has been compiled for 0374 * parallel. 0375 * 0376 * metadata_write_strategy: Integer field containing a code indicating the 0377 * desired metadata write strategy. The valid values of this field 0378 * are enumerated and discussed below: 0379 * 0380 * 0381 * H5AC_METADATA_WRITE_STRATEGY__PROCESS_0_ONLY: 0382 * 0383 * When metadata_write_strategy is set to this value, only process 0384 * zero is allowed to write dirty metadata to disk. All other 0385 * processes must retain dirty metadata until they are informed at 0386 * a sync point that the dirty metadata in question has been written 0387 * to disk. 0388 * 0389 * When the sync point is reached (or when there is a user generated 0390 * flush), process zero flushes sufficient entries to bring it into 0391 * compliance with its min clean size (or flushes all dirty entries in 0392 * the case of a user generated flush), broad casts the list of 0393 * entries just cleaned to all the other processes, and then exits 0394 * the sync point. 0395 * 0396 * Upon receipt of the broadcast, the other processes mark the indicated 0397 * entries as clean, and leave the sync point as well. 0398 * 0399 * 0400 * H5AC_METADATA_WRITE_STRATEGY__DISTRIBUTED: 0401 * 0402 * In the distributed metadata write strategy, process zero still makes 0403 * the decisions as to what entries should be flushed, but the actual 0404 * flushes are distributed across the processes in the computation to 0405 * the extent possible. 0406 * 0407 * In this strategy, when a sync point is triggered (either by dirty 0408 * metadata creation or manual flush), all processes enter a barrier. 0409 * 0410 * On the other side of the barrier, process 0 constructs an ordered 0411 * list of the entries to be flushed, and then broadcasts this list 0412 * to the caches in all the processes. 0413 * 0414 * All processes then scan the list of entries to be flushed, flushing 0415 * some, and marking the rest as clean. The algorithm for this purpose 0416 * ensures that each entry in the list is flushed exactly once, and 0417 * all are marked clean in each cache. 0418 * 0419 * Note that in the case of a flush of the cache, no message passing 0420 * is necessary, as all processes have the same list of dirty entries, 0421 * and all of these entries must be flushed. Thus in this case it is 0422 * sufficient for each process to sort its list of dirty entries after 0423 * leaving the initial barrier, and use this list as if it had been 0424 * received from process zero. 0425 * 0426 * To avoid possible messages from the past/future, all caches must 0427 * wait until all caches are done before leaving the sync point. 0428 * 0429 ****************************************************************************/ 0430 0431 #define H5AC__CURR_CACHE_CONFIG_VERSION 1 0432 #define H5AC__MAX_TRACE_FILE_NAME_LEN 1024 0433 0434 #define H5AC_METADATA_WRITE_STRATEGY__PROCESS_0_ONLY 0 0435 #define H5AC_METADATA_WRITE_STRATEGY__DISTRIBUTED 1 0436 0437 /** 0438 * H5AC_cache_config_t is a public structure intended for use in public APIs. 0439 * At least in its initial incarnation, it is basically a copy of \c struct 0440 * \c H5C_auto_size_ctl_t, minus the \c report_fcn field, and plus the 0441 * \c dirty_bytes_threshold field. 0442 * 0443 * The \c report_fcn field is omitted, as including it would require us to make 0444 * \c H5C_t structure public. 0445 * 0446 * The \c dirty_bytes_threshold field does not appear in \c H5C_auto_size_ctl_t, 0447 * as synchronization between caches on different processes is handled at the \c 0448 * H5AC level, not at the level of \c H5C. Note however that there is 0449 * considerable interaction between this value and the other fields in this 0450 * structure. 0451 * 0452 * Similarly, the \c open_trace_file, \c close_trace_file, and \c 0453 * trace_file_name fields do not appear in \c H5C_auto_size_ctl_t, as most trace 0454 * file issues are handled at the \c H5AC level. The one exception is storage 0455 * of the pointer to the trace file, which is handled by \c H5C. 0456 * 0457 * The structure is in H5ACpublic.h as we may wish to allow different 0458 * configuration options for metadata and raw data caches. 0459 */ 0460 0461 //! <!-- [H5AC_cache_config_t_snip] --> 0462 typedef struct H5AC_cache_config_t { 0463 /* general configuration fields: */ 0464 //! <!-- [H5AC_cache_config_t_general_snip] --> 0465 int version; 0466 /**< Integer field indicating the version of the H5AC_cache_config_t 0467 * in use. This field should be set to #H5AC__CURR_CACHE_CONFIG_VERSION 0468 * (defined in H5ACpublic.h). */ 0469 0470 hbool_t rpt_fcn_enabled; 0471 /**< Boolean flag indicating whether the adaptive cache resize report 0472 * function is enabled. This field should almost always be set to disabled 0473 * (0). Since resize algorithm activity is reported via stdout, it MUST be 0474 * set to disabled (0) on Windows machines.\n 0475 * The report function is not supported code, and can be expected to change 0476 * between versions of the library. Use it at your own risk. */ 0477 0478 hbool_t open_trace_file; 0479 /**< Boolean field indicating whether the 0480 * \ref H5AC_cache_config_t.trace_file_name "trace_file_name" 0481 * field should be used to open a trace file for the cache.\n 0482 * The trace file is a debugging feature that allows the capture 0483 * of top level metadata cache requests for purposes of debugging 0484 * and/or optimization. This field should normally be set to 0, as 0485 * trace file collection imposes considerable overhead.\n 0486 * This field should only be set to 1 when the 0487 * \ref H5AC_cache_config_t.trace_file_name "trace_file_name" 0488 * contains the full path of the desired trace file, and either 0489 * there is no open trace file on the cache, or the 0490 * \ref H5AC_cache_config_t.close_trace_file "close_trace_file" 0491 * field is also 1.\n 0492 * The trace file feature is unsupported unless used at the 0493 * direction of The HDF Group. It is intended to allow The HDF 0494 * Group to collect a trace of cache activity in cases of occult 0495 * failures and/or poor performance seen in the field, so as to aid 0496 * in reproduction in the lab. If you use it absent the direction 0497 * of The HDF Group, you are on your own. */ 0498 0499 hbool_t close_trace_file; 0500 /**< Boolean field indicating whether the current trace file 0501 *(if any) should be closed.\n 0502 * See the above comments on the \ref H5AC_cache_config_t.open_trace_file 0503 * "open_trace_file" field. This field should be set to 0 unless there is 0504 * an open trace file on the cache that you wish to close.\n 0505 * The trace file feature is unsupported unless used at the direction of 0506 * The HDF Group. It is intended to allow The HDF Group to collect a trace 0507 * of cache activity in cases of occult failures and/or poor performance 0508 * seen in the field, so as to aid in reproduction in the lab. If you use 0509 * it absent the direction of The HDF Group, you are on your own. */ 0510 0511 char trace_file_name[H5AC__MAX_TRACE_FILE_NAME_LEN + 1]; 0512 /**< Full path of the trace file to be opened if the 0513 * \ref H5AC_cache_config_t.open_trace_file "open_trace_file" field is set 0514 * to 1.\n 0515 * In the parallel case, an ascii representation of the MPI rank of the 0516 * process will be appended to the file name to yield a unique trace file 0517 * name for each process.\n 0518 * The length of the path must not exceed #H5AC__MAX_TRACE_FILE_NAME_LEN 0519 * characters.\n 0520 * The trace file feature is unsupported unless used at the direction of 0521 * The HDF Group. It is intended to allow The HDF Group to collect a trace 0522 * of cache activity in cases of occult failures and/or poor performance 0523 * seen in the field, so as to aid in reproduction in the lab. If you use 0524 * it absent the direction of The HDF Group, you are on your own. */ 0525 0526 hbool_t evictions_enabled; 0527 /**< A boolean flag indicating whether evictions from the metadata cache 0528 * are enabled. This flag is initially set to enabled (1).\n 0529 * In rare circumstances, the raw data throughput quirements may be so high 0530 * that the user wishes to postpone metadata writes so as to reserve I/O 0531 * throughput for raw data. The \p evictions_enabled field exists to allow 0532 * this. However, this is an extreme step, and you have no business doing 0533 * it unless you have read the User Guide section on metadata caching, and 0534 * have considered all other options carefully.\n 0535 * The \p evictions_enabled field may not be set to disabled (0) 0536 * unless all adaptive cache resizing code is disabled via the 0537 * \ref H5AC_cache_config_t.incr_mode "incr_mode", 0538 * \ref H5AC_cache_config_t.flash_incr_mode "flash_incr_mode", 0539 * \ref H5AC_cache_config_t.decr_mode "decr_mode" fields.\n 0540 * When this flag is set to disabled (\c 0), the metadata cache will not 0541 * attempt to evict entries to make space for new entries, and thus will 0542 * grow without bound.\n 0543 * Evictions will be re-enabled when this field is set back to \c 1. 0544 * This should be done as soon as possible. */ 0545 0546 hbool_t set_initial_size; 0547 /**< Boolean flag indicating whether the cache should be created 0548 * with a user specified initial size. */ 0549 0550 size_t initial_size; 0551 /**< If \ref H5AC_cache_config_t.set_initial_size "set_initial_size" 0552 * is set to 1, \p initial_size must contain he desired initial size in 0553 * bytes. This value must lie in the closed interval 0554 * [ \p min_size, \p max_size ]. (see below) */ 0555 0556 double min_clean_fraction; 0557 /**< This field specifies the minimum fraction of the cache 0558 * that must be kept either clean or empty.\n 0559 * The value must lie in the interval [0.0, 1.0]. 0.01 is a good place to 0560 * start in the serial case. In the parallel case, a larger value is needed 0561 * -- see the overview of the metadata cache in the 0562 * “Metadata Caching in HDF5” section of the -- <em>\ref UG</em> 0563 * for details. */ 0564 0565 size_t max_size; 0566 /**< Upper bound (in bytes) on the range of values that the 0567 * adaptive cache resize code can select as the maximum cache size. */ 0568 0569 size_t min_size; 0570 /**< Lower bound (in bytes) on the range of values that the 0571 * adaptive cache resize code can select as the minimum cache * size. */ 0572 0573 long int epoch_length; 0574 /**< Number of cache accesses between runs of the adaptive cache resize 0575 * code. 50,000 is a good starting number. */ 0576 //! <!-- [H5AC_cache_config_t_general_snip] --> 0577 0578 /* size increase control fields: */ 0579 //! <!-- [H5AC_cache_config_t_incr_snip] --> 0580 enum H5C_cache_incr_mode incr_mode; 0581 /**< Enumerated value indicating the operational mode of the automatic 0582 * cache size increase code. At present, only two values listed in 0583 * #H5C_cache_incr_mode are legal. */ 0584 0585 double lower_hr_threshold; 0586 /**< Hit rate threshold used by the hit rate threshold cache size 0587 * increment algorithm.\n 0588 * When the hit rate over an epoch is below this threshold and the cache 0589 * is full, the maximum size of the cache is multiplied by increment 0590 * (below), and then clipped as necessary to stay within \p max_size, and 0591 * possibly \p max_increment.\n 0592 * This field must lie in the interval [0.0, 1.0]. 0.8 or 0.9 is a good 0593 * place to start. */ 0594 0595 double increment; 0596 /**< Factor by which the hit rate threshold cache size increment 0597 * algorithm multiplies the current cache max size to obtain a tentative 0598 * new cache size.\n 0599 * The actual cache size increase will be clipped to satisfy the \p max_size 0600 * specified in the general configuration, and possibly max_increment 0601 * below.\n 0602 * The parameter must be greater than or equal to 1.0 -- 2.0 is a reasonable 0603 * value.\n 0604 * If you set it to 1.0, you will effectively disable cache size increases. 0605 */ 0606 0607 hbool_t apply_max_increment; 0608 /**< Boolean flag indicating whether an upper limit should be applied to 0609 * the size of cache size increases. */ 0610 0611 size_t max_increment; 0612 /**< Maximum number of bytes by which cache size can be increased in a 0613 * single step -- if applicable. */ 0614 0615 enum H5C_cache_flash_incr_mode flash_incr_mode; 0616 /**< Enumerated value indicating the operational mode of the flash cache 0617 * size increase code. At present, only two listed values in 0618 * #H5C_cache_flash_incr_mode are legal.*/ 0619 0620 double flash_multiple; 0621 /**< The factor by which the size of the triggering entry / entry size 0622 * increase is multiplied to obtain the initial cache size increment. This 0623 * increment may be reduced to reflect existing free space in the cache and 0624 * the \p max_size field above.\n 0625 * The parameter must lie in the interval [0.0, 1.0]. 0.1 or 0.05 is a good 0626 * place to start.\n 0627 * At present, this field must lie in the range [0.1, 10.0]. */ 0628 0629 double flash_threshold; 0630 /**< The factor by which the current maximum cache size is multiplied to 0631 * obtain the minimum size entry / entry size increase which may trigger a 0632 * flash cache size increase. \n 0633 * At present, this value must lie in the range [0.1, 1.0]. */ 0634 //! <!-- [H5AC_cache_config_t_incr_snip] --> 0635 0636 /* size decrease control fields: */ 0637 //! <!-- [H5AC_cache_config_t_decr_snip] --> 0638 enum H5C_cache_decr_mode decr_mode; 0639 /**< Enumerated value indicating the operational mode of the tomatic 0640 * cache size decrease code. At present, the values listed in 0641 * #H5C_cache_decr_mode are legal.*/ 0642 0643 double upper_hr_threshold; 0644 /**< Hit rate threshold for the hit rate threshold and ageout with hit 0645 * rate threshold cache size decrement algorithms.\n 0646 * When \p decr_mode is #H5C_decr__threshold, and the hit rate over a given 0647 * epoch exceeds the supplied threshold, the current maximum cache 0648 * size is multiplied by decrement to obtain a tentative new (and smaller) 0649 * maximum cache size.\n 0650 * When \p decr_mode is #H5C_decr__age_out_with_threshold, there is 0651 * no attempt to find and evict aged out entries unless the hit rate in 0652 * the previous epoch exceeded the supplied threshold.\n 0653 * This field must lie in the interval [0.0, 1.0].\n 0654 * For #H5C_incr__threshold, .9995 or .99995 is a good place to start.\n 0655 * For #H5C_decr__age_out_with_threshold, .999 might be more useful.*/ 0656 0657 double decrement; 0658 /**< In the hit rate threshold cache size decrease algorithm, this 0659 * parameter contains the factor by which the current max cache size is 0660 * multiplied to produce a tentative new cache size.\n 0661 * The actual cache size decrease will be clipped to satisfy the 0662 * \ref H5AC_cache_config_t.min_size "min_size" specified in the general 0663 * configuration, and possibly \ref H5AC_cache_config_t.max_decrement 0664 * "max_decrement".\n 0665 * The parameter must be be in the interval [0.0, 1.0].\n 0666 * If you set it to 1.0, you will effectively 0667 * disable cache size decreases. 0.9 is a reasonable starting point. */ 0668 0669 hbool_t apply_max_decrement; 0670 /**< Boolean flag indicating ether an upper limit should be applied to 0671 * the size of cache size decreases. */ 0672 0673 size_t max_decrement; 0674 /**< Maximum number of bytes by which the maximum cache size can be 0675 * decreased in any single step -- if applicable.*/ 0676 0677 int epochs_before_eviction; 0678 /**< In the ageout based cache size reduction algorithms, this field 0679 * contains the minimum number of epochs an entry must remain unaccessed in 0680 * cache before the cache size reduction algorithm tries to evict it. 3 is a 0681 * reasonable value. */ 0682 0683 hbool_t apply_empty_reserve; 0684 /**< Boolean flag indicating whether the ageout based decrement 0685 * algorithms will maintain a empty reserve when decreasing cache size. */ 0686 0687 double empty_reserve; 0688 /**< Empty reserve as a fraction maximum cache size if applicable.\n When 0689 * so directed, the ageout based algorithms will not decrease the maximum 0690 * cache size unless the empty reserve can be met.\n The parameter must lie 0691 * in the interval [0.0, 1.0]. 0.1 or 0.05 is a good place to start. */ 0692 //! <!-- [H5AC_cache_config_t_decr_snip] --> 0693 0694 /* parallel configuration fields: */ 0695 //! <!-- [H5AC_cache_config_t_parallel_snip] --> 0696 size_t dirty_bytes_threshold; 0697 /**< Threshold number of bytes of dirty metadata generation for 0698 * triggering synchronizations of the metadata caches serving the target 0699 * file in the parallel case.\n Synchronization occurs whenever the number 0700 * of bytes of dirty metadata created since the last synchronization exceeds 0701 * this limit.\n This field only applies to the parallel case. While it is 0702 * ignored elsewhere, it can still draw a value out of bounds error.\n It 0703 * must be consistent across all caches on any given file.\n By default, 0704 * this field is set to 256 KB. It shouldn't be more than half the current 0705 * max cache size times the min clean fraction. */ 0706 0707 int metadata_write_strategy; 0708 /**< Desired metadata write strategy. The valid values for this field 0709 * are:\n #H5AC_METADATA_WRITE_STRATEGY__PROCESS_0_ONLY: Specifies the only 0710 * process zero is allowed to write dirty metadata to disk.\n 0711 * #H5AC_METADATA_WRITE_STRATEGY__DISTRIBUTED: Specifies that process zero 0712 * still makes the decisions as to what entries should be flushed, but the 0713 * actual flushes are distributed across the processes in the computation to 0714 * the extent possible.\n The src/H5ACpublic.h include file in the HDF5 0715 * library has detailed information on each strategy. */ 0716 //! <!-- [H5AC_cache_config_t_parallel_snip] --> 0717 } H5AC_cache_config_t; 0718 //! <!-- [H5AC_cache_config_t_snip] --> 0719 0720 #define H5AC__CURR_CACHE_IMAGE_CONFIG_VERSION 1 0721 0722 #define H5AC__CACHE_IMAGE__ENTRY_AGEOUT__NONE -1 0723 #define H5AC__CACHE_IMAGE__ENTRY_AGEOUT__MAX 100 0724 0725 //! <!-- [H5AC_cache_image_config_t_snip] --> 0726 /** 0727 * H5AC_cache_image_config_t is a public structure intended for use in public 0728 * APIs. At least in its initial incarnation, it is a copy of \c struct \c 0729 * H5C_cache_image_ctl_t. 0730 */ 0731 0732 typedef struct H5AC_cache_image_config_t { 0733 int version; 0734 /**< Integer field containing the version number of this version of the \c 0735 * H5C_image_ctl_t structure. Any instance of \c H5C_image_ctl_t passed 0736 * to the cache must have a known version number, or an error will be 0737 * flagged. 0738 */ 0739 hbool_t generate_image; 0740 /**< Boolean flag indicating whether a cache image should be created on file 0741 * close. 0742 */ 0743 hbool_t save_resize_status; 0744 /**< Boolean flag indicating whether the cache image should include the 0745 * adaptive cache resize configuration and status. Note that this field 0746 * is ignored at present. 0747 */ 0748 int entry_ageout; 0749 /**< Integer field indicating the maximum number of times a 0750 * prefetched entry can appear in subsequent cache images. This field 0751 * exists to allow the user to avoid the buildup of infrequently used 0752 * entries in long sequences of cache images. 0753 * 0754 * The value of this field must lie in the range \ref 0755 * H5AC__CACHE_IMAGE__ENTRY_AGEOUT__NONE (-1) to \ref 0756 * H5AC__CACHE_IMAGE__ENTRY_AGEOUT__MAX (100). 0757 * 0758 * \ref H5AC__CACHE_IMAGE__ENTRY_AGEOUT__NONE means that no limit is 0759 * imposed on number of times a prefetched entry can appear in subsequent 0760 * cache images. 0761 * 0762 * A value of 0 prevents prefetched entries from being included in cache 0763 * images. 0764 * 0765 * Positive integers restrict prefetched entries to the specified number 0766 * of appearances. 0767 * 0768 * Note that the number of subsequent cache images that a prefetched entry 0769 * has appeared in is tracked in an 8 bit field. Thus, while \ref 0770 * H5AC__CACHE_IMAGE__ENTRY_AGEOUT__MAX can be increased from its current 0771 * value, any value in excess of 255 will be the functional equivalent of 0772 * \ref H5AC__CACHE_IMAGE__ENTRY_AGEOUT__NONE. 0773 */ 0774 } H5AC_cache_image_config_t; 0775 0776 //! <!-- [H5AC_cache_image_config_t_snip] --> 0777 0778 #endif
[ Source navigation ] | [ Diff markup ] | [ Identifier search ] | [ general search ] |
This page was automatically generated by the 2.3.7 LXR engine. The LXR team |
![]() ![]() |