User Tools

Site Tools


Table of Contents

HPIO

name:
HPIO
layers:
POSIX, MPI-IO
features:
metadata, tunable
parallel:
MPI
type:
synthetic
license:
GPLv2
webpage:
http://users.eecs.northwestern.edu/~aching/research_webpage/hpio.html

HPIO is a benchmark, which lets you test reads and writes to and from noncontiguous datalayouts.

Usage

You can set a great deal of flags to finetune the benchmark. To see them all run

mpirun -np 1 hpio -h

Example Output

For example runnning mpirun -np 4 ./hpio -d tmp -o output -a 0 -b 111 -n 1001 -m 111111 -O 11 -A 1 -v2 results in the following output

Click to display ⇲

Click to hide ⇱

Filling in parameters which have not been specified by the user...
** Initializing filename prefix to 't'
** Initializing default region count = 4096
** Initializing default region size = 8
** Initializing default region spacing = 128
** Initializing default reps = 1
** Initializing default rep max time = 600.000000
** Initializing to use default aggregator count
** Initializing to use 1 second wait time
** Initializing to use fsync
** Initializing to use INDIVIDUAL fsync method
** Initializing to use VECTOR pattern type
** Initializing use of partial generate file option if necessary
** Initializing to not keep created files that have been read
** Initializing to use persistent file realm size of 4194304
** Initializing to use different files per run
** Initializing cache size of 2 GBytes
** Initializing cache flush off
** Initializing enable resume off
** Initializing to not estimate space
** Verify forces wait time to 0.
** Warning: Two phase I/O using PFRs should use the 'same file mode' to open the file on the first repetition and close the file on the last repetition.  It operates on the same file (r0) with all reps on a given access pattern. PFRs may not work on your system if your MPI-IO implementation does not support it.
** Warning: Two phase I/O using PFRs will not perform well here since every repetition uses a new file.  For good PFR performance set the same file parameter to be 1.
#################### Experimental Settings ####################
procs                                   = 4
dir                                     = tmp
output_dir                              = output
default: region count                   = 4096
default: region size                    = 8
default: region spacing                 = 128
default: test (check single mode only)  = (null)
pattern datatype:                       = vector
rw: WRITE                               = 1
rw: READ                                = 1
bandwidth test: region_count            = 1
bandwidth test: region_size             = 1
bandwidth test: region_spacing          = 1
bandwidth test: single mode             = 0
check test: human                       = 0
check test: defined                     = 0
check test: single mode                 = 0
(M) Contig      | (F) Contig            = 1
(M) NonContig   | (F) Contig            = 0
(M) Contig      | (F) NonContig         = 0
(M) NonContig   | (F) NonContig         = 1
posix I/O                               = 1
data sieving I/O                        = 1
two phase I/O                           = 1
two phase I/O - PFRs                    = 1
list I/O                                = 1
datatype I/O                            = 1
reps                                    = 1
rep maximum time (seconds)              = 600.000000
average method                          = average all
verify                                  = verify file
aggregator method                       = 1
aggregator count                        = all
enable fsync                            = 1
fsync method                            = individual
same file                               = 0
generate files                          = partial (before reads)
keep files                              = 0
PFR size                                = 4194304
enable cache                            = 0
cache size (MBytes)                     = 2048
wait time (seconds between runs)        = 0.000000
check required space mode               = 0
enable resume test mode                 = 0
###############################################################

write | region_count | c-c | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.001 |  0.000 |  0.074 |  0.000 |  0.076 |385.506 |  0.838 | 2048
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |418.093 |  3.357 | 4096
  0.000 |  0.000 |  0.035 |  0.000 |  0.035 |1388.842 |  7.092 | 8192
  0.000 |  0.000 |  0.045 |  0.000 |  0.046 |1010.189 | 10.933 | 16384
  0.000 |  0.001 |  0.066 |  0.000 |  0.067 |904.140 | 14.999 | 32768
  0.000 |  0.003 |  0.066 |  0.000 |  0.069 |746.052 | 29.024 | 65536
  0.000 |  0.001 |  0.087 |  0.000 |  0.089 |3062.654 | 45.168 | 131072
  0.000 |  0.007 |  0.144 |  0.000 |  0.152 |1081.389 | 52.739 | 262144
  0.000 |  0.005 |  0.255 |  0.000 |  0.260 |3305.041 | 61.570 | 524288
  0.000 |  0.010 |  0.480 |  0.000 |  0.490 |3251.398 | 65.294 | 1048576

write | region_count | c-c | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.000 |  0.071 |  0.000 |  0.071 |315.456 |  0.884 | 2048
  0.000 |  0.000 |  0.038 |  0.000 |  0.039 |926.304 |  3.257 | 4096
  0.000 |  0.000 |  0.043 |  0.000 |  0.044 |908.645 |  5.746 | 8192
  0.000 |  0.001 |  0.037 |  0.000 |  0.038 |698.352 | 13.267 | 16384
  0.000 |  0.001 |  0.049 |  0.000 |  0.051 |1035.119 | 19.900 | 32768
  0.000 |  0.001 |  0.066 |  0.000 |  0.068 |1538.347 | 29.742 | 65536
  0.000 |  0.003 |  0.099 |  0.000 |  0.102 |1554.597 | 39.306 | 131072
  0.000 |  0.002 |  0.150 |  0.000 |  0.153 |3528.702 | 52.506 | 262144
  0.000 |  0.015 |  0.245 |  0.000 |  0.260 |1095.726 | 61.516 | 524288
  0.000 |  0.040 |  0.440 |  0.000 |  0.481 |797.946 | 66.586 | 1048576

write | region_count | c-c | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |238.530 |  1.704 | 2048
  0.000 |  0.000 |  0.037 |  0.000 |  0.038 |422.472 |  3.341 | 4096
  0.000 |  0.000 |  0.035 |  0.000 |  0.036 |617.172 |  7.065 | 8192
  0.000 |  0.001 |  0.047 |  0.000 |  0.048 |595.274 | 10.453 | 16384
  0.000 |  0.001 |  0.049 |  0.000 |  0.051 |669.268 | 19.687 | 32768
  0.000 |  0.004 |  0.073 |  0.000 |  0.077 |557.864 | 25.990 | 65536
  0.000 |  0.007 |  0.103 |  0.000 |  0.109 |615.204 | 36.666 | 131072
  0.000 |  0.013 |  0.152 |  0.000 |  0.166 |625.339 | 48.409 | 262144
  0.000 |  0.025 |  0.257 |  0.000 |  0.282 |642.079 | 56.752 | 524288
  0.000 |  0.050 |  0.466 |  0.000 |  0.516 |639.616 | 62.060 | 1048576

write | region_count | c-c | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.000 |  0.031 |  0.000 |  0.032 |328.913 |  1.978 | 2048
  0.000 |  0.000 |  0.030 |  0.000 |  0.030 |510.504 |  4.164 | 4096
  0.000 |  0.000 |  0.043 |  0.000 |  0.044 |624.896 |  5.721 | 8192
  0.000 |  0.001 |  0.045 |  0.000 |  0.046 |596.629 | 10.847 | 16384
  0.000 |  0.001 |  0.049 |  0.000 |  0.051 |676.064 | 19.719 | 32768
  0.000 |  0.004 |  0.065 |  0.000 |  0.069 |565.613 | 29.292 | 65536
  0.000 |  0.006 |  0.094 |  0.000 |  0.101 |616.991 | 39.638 | 131072
  0.000 |  0.013 |  0.153 |  0.000 |  0.166 |636.344 | 48.374 | 262144
  0.000 |  0.025 |  0.256 |  0.000 |  0.282 |638.087 | 56.899 | 524288
  0.000 |  0.050 |  0.463 |  0.000 |  0.513 |646.241 | 62.400 | 1048576

write | region_count | c-c | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.000 |  0.061 |  0.000 |  0.061 |372.364 |  1.026 | 2048
  0.000 |  0.000 |  0.046 |  0.000 |  0.047 |664.497 |  2.686 | 4096
  0.000 |  0.000 |  0.044 |  0.000 |  0.044 |3246.365 |  5.732 | 8192
  0.000 |  0.001 |  0.037 |  0.000 |  0.038 |740.781 | 13.271 | 16384
  0.000 |  0.001 |  0.049 |  0.000 |  0.051 |758.189 | 19.679 | 32768
  0.000 |  0.001 |  0.070 |  0.000 |  0.071 |3629.861 | 28.462 | 65536
  0.000 |  0.001 |  0.092 |  0.000 |  0.093 |2998.609 | 43.072 | 131072
  0.000 |  0.005 |  0.149 |  0.000 |  0.154 |1694.155 | 52.170 | 262144
  0.000 |  0.005 |  0.280 |  0.000 |  0.285 |3486.718 | 56.199 | 524288
  0.000 |  0.009 |  0.456 |  0.000 |  0.465 |3434.611 | 68.825 | 1048576

write | region_count | c-c | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.000 |  0.071 |  0.000 |  0.072 |771.012 |  0.873 | 2048
  0.000 |  0.000 |  0.038 |  0.000 |  0.038 |1042.322 |  3.273 | 4096
  0.000 |  0.000 |  0.045 |  0.000 |  0.045 |1275.640 |  5.525 | 8192
  0.000 |  0.000 |  0.046 |  0.000 |  0.046 |3449.263 | 10.913 | 16384
  0.000 |  0.000 |  0.050 |  0.000 |  0.051 |3634.579 | 19.807 | 32768
  0.000 |  0.001 |  0.067 |  0.000 |  0.068 |2915.748 | 29.670 | 65536
  0.000 |  0.001 |  0.093 |  0.000 |  0.094 |3223.288 | 42.525 | 131072
  0.000 |  0.007 |  0.148 |  0.000 |  0.155 |1074.567 | 51.609 | 262144
  0.000 |  0.005 |  0.257 |  0.000 |  0.262 |3212.795 | 61.116 | 524288
  0.000 |  0.009 |  0.481 |  0.000 |  0.491 |3430.485 | 65.212 | 1048576

write | region_count | nc-nc | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.015 |  0.052 |  0.000 |  0.068 |  4.174 |  0.929 | 2048
  0.000 |  0.033 |  0.061 |  0.000 |  0.094 |  3.788 |  1.333 | 4096
  0.000 |  0.057 |  0.092 |  0.000 |  0.150 |  4.362 |  1.672 | 8192
  0.001 |  0.130 |  0.141 |  0.000 |  0.272 |  3.849 |  1.846 | 16384
  0.001 |  0.251 |  0.267 |  0.000 |  0.519 |  3.981 |  1.931 | 32768
  0.002 |  0.519 |  0.475 |  0.000 |  0.996 |  3.855 |  2.012 | 65536
  0.005 |  1.033 |  0.941 |  0.000 |  1.979 |  3.871 |  2.026 | 131072
  0.010 |  2.045 |  1.816 |  0.000 |  3.871 |  3.912 |  2.072 | 262144
  0.019 |  4.195 |  3.547 |  0.000 |  7.762 |  3.814 |  2.066 | 524288
  0.036 |  8.185 |  7.583 |  0.000 | 15.804 |  3.910 |  2.029 | 1048576

write | region_count | nc-nc | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.003 |  0.061 |  0.000 |  0.065 | 19.291 |  0.966 | 2048
  0.000 |  0.010 |  0.128 |  0.000 |  0.138 | 12.448 |  0.907 | 4096
  0.000 |  0.017 |  0.100 |  0.000 |  0.118 | 14.415 |  2.122 | 8192
  0.001 |  0.041 |  0.170 |  0.000 |  0.213 | 12.097 |  2.361 | 16384
  0.001 |  0.072 |  0.278 |  0.000 |  0.351 | 13.877 |  2.859 | 32768
  0.002 |  0.149 |  0.486 |  0.000 |  0.637 | 13.442 |  3.149 | 65536
  0.005 |  0.226 |  1.615 |  0.000 |  1.846 | 17.688 |  2.173 | 131072
  0.010 |  0.437 |  3.288 |  0.000 |  3.735 | 18.294 |  2.148 | 262144
  0.019 |  1.191 |  3.697 |  0.000 |  4.907 | 13.438 |  3.274 | 524288
  0.036 |  2.094 |  7.432 |  0.000 |  9.562 | 15.284 |  3.359 | 1048576

write | region_count | nc-nc | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.003 |  0.063 |  0.000 |  0.066 | 19.962 |  0.947 | 2048
  0.000 |  0.007 |  0.072 |  0.000 |  0.079 | 17.723 |  1.587 | 4096
  0.000 |  0.014 |  0.093 |  0.000 |  0.108 | 17.278 |  2.334 | 8192
  0.001 |  0.029 |  0.200 |  0.000 |  0.229 | 17.444 |  2.191 | 16384
  0.001 |  0.057 |  0.266 |  0.000 |  0.324 | 17.523 |  3.097 | 32768
  0.002 |  0.117 |  0.495 |  0.000 |  0.614 | 17.128 |  3.267 | 65536
  0.005 |  0.235 |  0.946 |  0.000 |  1.187 | 17.009 |  3.387 | 131072
  0.010 |  0.506 |  1.872 |  0.005 |  2.393 | 15.820 |  3.364 | 262144
  0.019 |  1.063 |  3.696 |  0.000 |  4.778 | 15.050 |  3.362 | 524288
  0.036 |  2.325 |  7.537 |  0.000 |  9.898 | 13.763 |  3.245 | 1048576

write | region_count | nc-nc | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.003 |  0.057 |  0.000 |  0.060 | 18.174 |  1.043 | 2048
  0.000 |  0.007 |  0.072 |  0.000 |  0.079 | 17.758 |  1.585 | 4096
  0.001 |  0.015 |  0.100 |  0.000 |  0.115 | 17.188 |  2.177 | 8192
  0.001 |  0.029 |  0.157 |  0.000 |  0.186 | 17.474 |  2.697 | 16384
  0.001 |  0.057 |  0.280 |  0.000 |  0.339 | 17.529 |  2.963 | 32768
  0.002 |  0.113 |  0.494 |  0.000 |  0.609 | 17.761 |  3.296 | 65536
  0.005 |  0.248 |  0.952 |  0.010 |  1.215 | 16.159 |  3.333 | 131072
  0.010 |  0.489 |  1.854 |  0.002 |  2.355 | 16.370 |  3.415 | 262144
  0.019 |  1.064 |  3.715 |  0.010 |  4.808 | 15.043 |  3.348 | 524288
  0.037 |  2.322 |  7.532 |  0.002 |  9.892 | 13.782 |  3.248 | 1048576

write | region_count | nc-nc | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.013 |  0.089 |  0.000 |  0.103 |  4.670 |  0.611 | 2048
  0.000 |  0.032 |  0.067 |  0.000 |  0.100 |  3.880 |  1.261 | 4096
  0.000 |  0.065 |  0.102 |  0.000 |  0.168 |  3.828 |  1.492 | 8192
  0.001 |  0.123 |  0.182 |  0.000 |  0.307 |  4.051 |  1.636 | 16384
  0.001 |  0.233 |  0.254 |  0.000 |  0.489 |  4.283 |  2.050 | 32768
  0.002 |  0.512 |  0.487 |  0.000 |  1.002 |  3.904 |  2.001 | 65536
  0.005 |  1.004 |  0.941 |  0.000 |  1.950 |  3.985 |  2.057 | 131072
  0.010 |  2.093 |  1.772 |  0.000 |  3.876 |  3.822 |  2.070 | 262144
  0.019 |  4.153 |  3.465 |  0.000 |  7.637 |  3.853 |  2.100 | 524288
  0.037 |  8.492 |  6.624 |  0.000 | 15.153 |  3.768 |  2.117 | 1048576

write | region_count | nc-nc | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.013 |  0.084 |  0.000 |  0.097 |  4.758 |  0.647 | 2048
  0.000 |  0.033 |  0.094 |  0.000 |  0.127 |  3.804 |  0.985 | 4096
  0.000 |  0.098 |  0.052 |  0.000 |  0.150 |  2.559 |  1.673 | 8192
  0.015 |  0.128 |  0.130 |  0.000 |  0.272 |  3.919 |  1.944 | 16384
  0.001 |  0.252 |  0.241 |  0.000 |  0.494 |  3.972 |  2.029 | 32768
  0.002 |  0.874 |  0.125 |  0.000 |  1.002 |  2.289 |  2.002 | 65536
  0.005 |  0.970 |  0.800 |  0.000 |  1.775 |  4.125 |  2.261 | 131072
  0.010 |  2.112 |  1.522 |  0.000 |  3.644 |  3.788 |  2.201 | 262144
  0.019 |  4.209 |  3.032 |  0.000 |  7.261 |  3.801 |  2.210 | 524288
  0.037 |  7.988 |  6.627 |  0.000 | 14.652 |  4.006 |  2.189 | 1048576

write | region_size | c-c | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.017 |  0.000 |  0.042 |  0.000 |  0.059 |875.272 |  2.991 | 8
  0.000 |  0.000 |  0.035 |  0.000 |  0.035 |764.826 |  7.089 | 16
  0.000 |  0.000 |  0.037 |  0.000 |  0.038 |1299.351 | 13.216 | 32
  0.000 |  0.001 |  0.042 |  0.000 |  0.043 |905.897 | 23.451 | 64
  0.000 |  0.001 |  0.067 |  0.000 |  0.069 |1375.633 | 29.053 | 128
  0.000 |  0.005 |  0.080 |  0.000 |  0.085 |757.129 | 47.070 | 256
  0.000 |  0.005 |  0.142 |  0.000 |  0.147 |1684.967 | 54.539 | 512
  0.000 |  0.010 |  0.219 |  0.000 |  0.229 |1672.912 | 69.920 | 1024
  0.000 |  0.053 |  0.365 |  0.000 |  0.418 |600.341 | 76.544 | 2048
  0.000 |  0.058 |  0.719 |  0.000 |  0.778 |1096.152 | 82.290 | 4096

write | region_size | c-c | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.039 |  0.000 |  0.040 |573.619 |  3.187 | 8
  0.000 |  0.000 |  0.035 |  0.000 |  0.035 |976.328 |  7.088 | 16
  0.000 |  0.000 |  0.037 |  0.000 |  0.038 |1033.080 | 13.173 | 32
  0.000 |  0.001 |  0.068 |  0.000 |  0.070 |723.530 | 14.359 | 64
  0.000 |  0.001 |  0.070 |  0.000 |  0.071 |3257.712 | 28.430 | 128
  0.000 |  0.003 |  0.090 |  0.000 |  0.092 |1566.208 | 43.448 | 256
  0.000 |  0.008 |  0.133 |  0.000 |  0.141 |1056.101 | 56.767 | 512
  0.000 |  0.015 |  0.215 |  0.000 |  0.230 |1077.864 | 69.627 | 1024
  0.000 |  0.009 |  0.395 |  0.000 |  0.404 |3433.908 | 79.239 | 2048
  0.000 |  0.019 |  0.769 |  0.000 |  0.787 |3448.466 | 81.303 | 4096

write | region_size | c-c | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.039 |  0.000 |  0.040 |302.532 |  3.142 | 8
  0.000 |  0.000 |  0.035 |  0.000 |  0.036 |589.419 |  7.053 | 16
  0.000 |  0.001 |  0.046 |  0.000 |  0.047 |620.276 | 10.792 | 32
  0.000 |  0.001 |  0.042 |  0.000 |  0.043 |669.268 | 23.201 | 64
  0.000 |  0.004 |  0.066 |  0.000 |  0.070 |515.334 | 28.788 | 128
  0.000 |  0.008 |  0.086 |  0.000 |  0.095 |501.186 | 42.375 | 256
  0.000 |  0.013 |  0.133 |  0.000 |  0.146 |623.688 | 55.034 | 512
  0.000 |  0.028 |  0.221 |  0.000 |  0.249 |578.874 | 64.291 | 1024
  0.000 |  0.049 |  0.394 |  0.000 |  0.443 |652.823 | 72.211 | 2048
  0.000 |  0.100 |  0.750 |  0.000 |  0.851 |636.969 | 75.210 | 4096

write | region_size | c-c | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |335.223 |  3.437 | 8
  0.000 |  0.000 |  0.043 |  0.000 |  0.044 |540.225 |  5.735 | 16
  0.000 |  0.001 |  0.037 |  0.000 |  0.038 |594.599 | 13.226 | 32
  0.000 |  0.001 |  0.050 |  0.000 |  0.052 |674.759 | 19.476 | 64
  0.000 |  0.005 |  0.077 |  0.010 |  0.092 |370.440 | 24.403 | 128
  0.000 |  0.008 |  0.086 |  0.000 |  0.095 |505.947 | 42.426 | 256
  0.000 |  0.013 |  0.130 |  0.000 |  0.143 |635.176 | 55.996 | 512
  0.000 |  0.027 |  0.222 |  0.000 |  0.250 |584.648 | 64.141 | 1024
  0.000 |  0.050 |  0.403 |  0.000 |  0.453 |643.786 | 70.645 | 2048
  0.000 |  0.100 |  0.748 |  0.000 |  0.848 |640.039 | 75.511 | 4096

write | region_size | c-c | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.044 |  0.000 |  0.045 |2319.858 |  2.825 | 8
  0.000 |  0.000 |  0.035 |  0.000 |  0.035 |876.736 |  7.094 | 16
  0.000 |  0.001 |  0.037 |  0.000 |  0.038 |729.952 | 13.175 | 32
  0.000 |  0.000 |  0.040 |  0.000 |  0.041 |3155.985 | 24.524 | 64
  0.000 |  0.003 |  0.065 |  0.000 |  0.069 |729.698 | 29.310 | 128
  0.000 |  0.003 |  0.081 |  0.000 |  0.084 |1546.145 | 47.739 | 256
  0.000 |  0.005 |  0.138 |  0.000 |  0.144 |1586.948 | 55.784 | 512
  0.000 |  0.020 |  0.205 |  0.000 |  0.225 |814.537 | 71.075 | 1024
  0.000 |  0.040 |  0.435 |  0.000 |  0.475 |808.594 | 67.363 | 2048
  0.000 |  0.019 |  0.771 |  0.000 |  0.791 |3296.923 | 80.970 | 4096

write | region_size | c-c | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.043 |  0.000 |  0.044 |2193.674 |  2.876 | 8
  0.000 |  0.000 |  0.035 |  0.000 |  0.036 |915.787 |  7.081 | 16
  0.000 |  0.000 |  0.054 |  0.000 |  0.054 |3377.056 |  9.224 | 32
  0.000 |  0.001 |  0.050 |  0.000 |  0.051 |1468.080 | 19.597 | 64
  0.000 |  0.002 |  0.058 |  0.000 |  0.060 |916.988 | 33.319 | 128
  0.000 |  0.001 |  0.090 |  0.000 |  0.092 |3286.428 | 43.750 | 256
  0.000 |  0.008 |  0.136 |  0.000 |  0.144 |1066.100 | 55.563 | 512
  0.000 |  0.010 |  0.231 |  0.000 |  0.241 |1644.744 | 66.489 | 1024
  0.000 |  0.009 |  0.428 |  0.000 |  0.438 |3392.335 | 73.185 | 2048
  0.000 |  0.039 |  0.728 |  0.000 |  0.768 |1639.311 | 83.410 | 4096

write | region_size | nc-nc | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.033 |  0.060 |  0.000 |  0.094 |  3.778 |  1.339 | 8
  0.000 |  0.028 |  0.065 |  0.000 |  0.093 |  9.075 |  2.709 | 16
  0.000 |  0.033 |  0.064 |  0.000 |  0.097 | 15.078 |  5.162 | 32
  0.000 |  0.053 |  0.062 |  0.000 |  0.115 | 18.792 |  8.698 | 64
  0.000 |  0.026 |  0.091 |  0.000 |  0.117 | 77.054 | 17.093 | 128
  0.000 |  0.039 |  0.106 |  0.000 |  0.146 |101.494 | 27.483 | 256
  0.000 |  0.049 |  0.137 |  0.000 |  0.186 |164.247 | 43.023 | 512
  0.000 |  0.051 |  0.448 |  0.000 |  0.499 |312.646 | 32.076 | 1024
  0.000 |  0.580 |  1.189 |  0.000 |  1.770 | 55.135 | 18.081 | 2048
  0.000 |  0.155 |  1.153 |  0.000 |  1.309 |411.999 | 48.909 | 4096

write | region_size | nc-nc | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.009 |  0.153 |  0.000 |  0.163 | 13.458 |  0.770 | 8
  0.000 |  0.130 |  0.194 |  0.000 |  0.324 |  1.924 |  0.772 | 16
  0.000 |  0.012 |  0.142 |  0.000 |  0.154 | 42.499 |  3.261 | 32
  0.000 |  0.015 |  0.150 |  0.000 |  0.165 | 68.720 |  6.094 | 64
  0.001 |  0.018 |  0.226 |  0.000 |  0.245 |108.131 |  8.178 | 128
  0.001 |  0.028 |  0.217 |  0.000 |  0.247 |140.562 | 16.282 | 256
  0.000 |  0.050 |  0.366 |  0.000 |  0.416 |161.502 | 19.271 | 512
  0.000 |  0.071 |  0.382 |  0.000 |  0.453 |224.454 | 35.336 | 1024
  0.000 |  0.147 |  0.421 |  0.000 |  0.569 |217.804 | 56.329 | 2048
  0.001 |  0.311 |  1.034 |  0.000 |  1.347 |205.753 | 47.565 | 4096

write | region_size | nc-nc | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.008 |  0.061 |  0.000 |  0.069 | 15.889 |  1.821 | 8
  0.000 |  0.007 |  0.067 |  0.000 |  0.075 | 34.070 |  3.372 | 16
  0.000 |  0.008 |  0.068 |  0.000 |  0.077 | 59.283 |  6.527 | 32
  0.000 |  0.011 |  0.069 |  0.000 |  0.081 | 90.179 | 12.474 | 64
  0.000 |  0.015 |  0.090 |  0.000 |  0.105 |132.899 | 19.054 | 128
  0.000 |  0.022 |  0.107 |  0.000 |  0.129 |179.655 | 31.061 | 256
  0.000 |  0.036 |  0.161 |  0.000 |  0.197 |223.845 | 40.684 | 512
  0.000 |  0.065 |  0.237 |  0.000 |  0.303 |244.618 | 52.842 | 1024
  0.000 |  0.116 |  0.420 |  0.000 |  0.537 |275.241 | 59.697 | 2048
  0.000 |  0.223 |  0.775 |  0.010 |  1.008 |286.543 | 64.135 | 4096

write | region_size | nc-nc | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.008 |  0.074 |  0.010 |  0.093 | 15.505 |  1.516 | 8
  0.000 |  0.007 |  0.064 |  0.000 |  0.071 | 34.143 |  3.528 | 16
  0.000 |  0.008 |  0.069 |  0.000 |  0.078 | 59.409 |  6.467 | 32
  0.000 |  0.011 |  0.071 |  0.000 |  0.083 | 89.542 | 12.165 | 64
  0.000 |  0.015 |  0.090 |  0.000 |  0.105 |133.655 | 19.046 | 128
  0.000 |  0.022 |  0.132 |  0.000 |  0.155 |179.872 | 25.908 | 256
  0.000 |  0.036 |  0.162 |  0.000 |  0.199 |221.657 | 40.361 | 512
  0.000 |  0.065 |  0.242 |  0.000 |  0.308 |246.086 | 52.061 | 1024
  0.000 |  0.117 |  0.424 |  0.002 |  0.542 |274.487 | 59.247 | 2048
  0.000 |  0.224 |  0.850 |  0.002 |  1.077 |285.506 | 59.555 | 4096

write | region_size | nc-nc | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.029 |  0.073 |  0.000 |  0.102 |  4.268 |  1.226 | 8
  0.000 |  0.033 |  0.066 |  0.000 |  0.100 |  7.467 |  2.517 | 16
  0.000 |  0.033 |  0.063 |  0.000 |  0.096 | 15.086 |  5.207 | 32
  0.000 |  0.032 |  0.080 |  0.000 |  0.113 | 30.835 |  8.879 | 64
  0.000 |  0.036 |  0.079 |  0.000 |  0.116 | 55.256 | 17.335 | 128
  0.000 |  0.034 |  0.109 |  0.000 |  0.143 |117.003 | 28.024 | 256
  0.000 |  0.048 |  0.154 |  0.000 |  0.202 |167.898 | 39.606 | 512
  0.001 |  0.058 |  0.242 |  0.000 |  0.301 |273.598 | 53.257 | 1024
  0.000 |  0.070 |  0.507 |  0.000 |  0.577 |457.436 | 55.494 | 2048
  0.000 |  0.253 |  1.062 |  0.000 |  1.316 |252.862 | 48.657 | 4096

write | region_size | nc-nc | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.047 |  0.107 |  0.000 |  0.154 |  2.679 |  0.812 | 8
  0.000 |  0.034 |  0.094 |  0.000 |  0.128 |  7.380 |  1.959 | 16
  0.000 |  0.028 |  0.072 |  0.000 |  0.101 | 17.982 |  4.999 | 32
  0.000 |  0.033 |  0.083 |  0.000 |  0.116 | 30.441 |  8.664 | 64
  0.000 |  0.030 |  0.092 |  0.000 |  0.123 | 66.689 | 16.346 | 128
  0.000 |  0.037 |  0.107 |  0.000 |  0.144 |106.729 | 27.771 | 256
  0.000 |  0.047 |  0.147 |  0.000 |  0.195 |171.494 | 41.213 | 512
  0.000 |  0.067 |  0.381 |  0.000 |  0.449 |239.263 | 35.704 | 1024
  0.001 |  0.086 |  0.802 |  0.000 |  0.889 |371.191 | 36.028 | 2048
  0.000 |  0.149 |  0.712 |  0.000 |  0.861 |430.090 | 74.382 | 4096

write | region_spacing | c-c | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.032 |  0.000 |  0.033 |506.069 |  3.866 | 8
  0.000 |  0.000 |  0.028 |  0.000 |  0.028 |2912.711 |  4.430 | 16
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |2557.502 |  4.385 | 32
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |1034.099 |  4.403 | 64
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |1076.567 |  3.401 | 128
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |985.504 |  3.401 | 256
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |1059.168 |  3.407 | 512
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |766.503 |  3.404 | 1024
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |701.858 |  3.409 | 2048
  0.000 |  0.000 |  0.038 |  0.000 |  0.039 |1040.254 |  3.258 | 4096

write | region_spacing | c-c | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |1127.501 |  3.405 | 8
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |992.970 |  4.416 | 16
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |545.565 |  4.407 | 32
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |1000.550 |  4.389 | 64
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |1016.062 |  4.384 | 128
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |800.440 |  4.398 | 256
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |512.500 |  4.399 | 512
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |2449.944 |  3.407 | 1024
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |776.723 |  3.399 | 2048
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |2833.989 |  3.401 | 4096

write | region_spacing | c-c | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |563.145 |  4.396 | 8
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |527.453 |  4.384 | 16
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |536.082 |  4.402 | 32
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |568.026 |  4.413 | 64
  0.000 |  0.000 |  0.030 |  0.000 |  0.030 |502.192 |  4.161 | 128
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |541.061 |  3.399 | 256
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |453.144 |  3.440 | 512
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |518.584 |  4.399 | 1024
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |516.540 |  4.381 | 2048
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |518.584 |  3.400 | 4096

write | region_spacing | c-c | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |538.837 |  3.399 | 8
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |545.565 |  3.408 | 16
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |541.061 |  3.410 | 32
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |512.500 |  4.404 | 64
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |514.512 |  4.394 | 128
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |536.082 |  4.410 | 256
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |553.046 |  4.383 | 512
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |434.013 |  4.405 | 1024
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |516.540 |  4.410 | 2048
  0.000 |  0.000 |  0.045 |  0.000 |  0.045 |503.639 |  2.786 | 4096

write | region_spacing | c-c | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.038 |  0.000 |  0.038 |1096.837 |  3.286 | 8
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |2661.360 |  3.407 | 16
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |941.271 |  3.418 | 32
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |2157.564 |  3.411 | 64
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |748.983 |  4.399 | 128
  0.000 |  0.000 |  0.028 |  0.000 |  0.028 |1008.246 |  4.433 | 256
  0.000 |  0.000 |  0.036 |  0.000 |  0.036 |1024.000 |  3.476 | 512
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |757.642 |  4.414 | 1024
  0.000 |  0.000 |  0.028 |  0.000 |  0.028 |747.914 |  4.427 | 2048
  0.000 |  0.000 |  0.030 |  0.000 |  0.030 |992.970 |  4.159 | 4096

write | region_spacing | c-c | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |2595.485 |  3.438 | 8
  0.000 |  0.000 |  0.028 |  0.000 |  0.029 |682.667 |  4.394 | 16
  0.000 |  0.000 |  0.028 |  0.000 |  0.028 |657.827 |  4.438 | 32
  0.000 |  0.000 |  0.144 |  0.000 |  0.145 |490.447 |  0.865 | 64
  0.000 |  0.000 |  0.073 |  0.000 |  0.073 |634.731 |  1.712 | 128
  0.000 |  0.000 |  0.041 |  0.000 |  0.041 |651.289 |  3.035 | 256
  0.000 |  0.000 |  0.040 |  0.000 |  0.040 |654.542 |  3.123 | 512
  0.000 |  0.000 |  0.037 |  0.000 |  0.037 |558.347 |  3.397 | 1024
  0.000 |  0.000 |  0.036 |  0.000 |  0.037 |493.680 |  3.406 | 2048
  0.000 |  0.000 |  0.029 |  0.000 |  0.030 |932.897 |  4.220 | 4096

write | region_spacing | nc-nc | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.029 |  0.056 |  0.000 |  0.085 |  4.336 |  1.470 | 8
  0.000 |  0.027 |  0.081 |  0.000 |  0.109 |  4.595 |  1.155 | 16
  0.000 |  0.026 |  0.040 |  0.000 |  0.066 |  4.782 |  1.898 | 32
  0.000 |  0.032 |  0.083 |  0.000 |  0.116 |  3.910 |  1.085 | 64
  0.000 |  0.031 |  0.063 |  0.000 |  0.094 |  4.025 |  1.336 | 128
  0.000 |  0.101 |  0.058 |  0.000 |  0.159 |  1.243 |  0.791 | 256
  0.000 |  0.033 |  0.131 |  0.000 |  0.165 |  3.787 |  0.760 | 512
  0.000 |  0.056 |  0.229 |  0.000 |  0.285 |  2.248 |  0.440 | 1024
  0.000 |  0.046 |  0.425 |  0.000 |  0.471 |  2.745 |  0.266 | 2048
  0.000 |  0.117 |  0.742 |  0.000 |  0.859 |  1.068 |  0.146 | 4096

write | region_spacing | nc-nc | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.004 |  0.037 |  0.000 |  0.041 | 34.490 |  3.113 | 8
  0.000 |  0.001 |  0.049 |  0.000 |  0.050 |103.819 |  2.496 | 16
  0.000 |  0.002 |  0.047 |  0.000 |  0.049 | 74.494 |  2.545 | 32
  0.000 |  0.006 |  0.058 |  0.000 |  0.064 | 21.434 |  1.973 | 64
  0.000 |  0.007 |  0.066 |  0.000 |  0.074 | 16.824 |  1.697 | 128
  0.000 |  0.019 |  0.091 |  0.000 |  0.111 |  6.469 |  1.129 | 256
  0.000 |  0.030 |  0.139 |  0.000 |  0.170 |  4.127 |  0.738 | 512
  0.001 |  0.067 |  0.224 |  0.000 |  0.291 |  1.876 |  0.430 | 1024
  0.000 |  0.116 |  0.409 |  0.000 |  0.526 |  1.080 |  0.238 | 2048
  0.001 |  0.260 |  0.766 |  0.000 |  1.027 |  0.480 |  0.122 | 4096

write | region_spacing | nc-nc | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.003 |  0.044 |  0.000 |  0.047 | 49.289 |  2.659 | 8
  0.000 |  0.003 |  0.036 |  0.000 |  0.039 | 45.906 |  3.193 | 16
  0.000 |  0.003 |  0.043 |  0.000 |  0.047 | 36.971 |  2.683 | 32
  0.000 |  0.005 |  0.056 |  0.000 |  0.061 | 27.233 |  2.056 | 64
  0.000 |  0.009 |  0.065 |  0.000 |  0.074 | 14.540 |  1.696 | 128
  0.000 |  0.011 |  0.088 |  0.000 |  0.100 | 10.910 |  1.253 | 256
  0.000 |  0.020 |  0.131 |  0.000 |  0.151 |  6.324 |  0.831 | 512
  0.000 |  0.035 |  0.225 |  0.000 |  0.261 |  3.532 |  0.480 | 1024
  0.000 |  0.068 |  0.422 |  0.000 |  0.491 |  1.845 |  0.255 | 2048
  0.001 |  0.155 |  0.751 |  0.002 |  0.909 |  0.807 |  0.138 | 4096

write | region_spacing | nc-nc | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.003 |  0.041 |  0.000 |  0.044 | 47.314 |  2.843 | 8
  0.000 |  0.003 |  0.036 |  0.000 |  0.039 | 47.064 |  3.202 | 16
  0.000 |  0.003 |  0.043 |  0.000 |  0.047 | 36.273 |  2.683 | 32
  0.000 |  0.005 |  0.047 |  0.000 |  0.052 | 27.251 |  2.419 | 64
  0.000 |  0.012 |  0.062 |  0.000 |  0.074 | 10.669 |  1.699 | 128
  0.000 |  0.011 |  0.088 |  0.000 |  0.100 | 11.067 |  1.257 | 256
  0.000 |  0.020 |  0.131 |  0.000 |  0.151 |  6.291 |  0.829 | 512
  0.000 |  0.036 |  0.248 |  0.000 |  0.284 |  3.515 |  0.440 | 1024
  0.000 |  0.070 |  0.398 |  0.010 |  0.478 |  1.798 |  0.267 | 2048
  0.001 |  0.135 |  0.757 |  0.000 |  0.893 |  0.927 |  0.140 | 4096

write | region_spacing | nc-nc | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.027 |  0.051 |  0.000 |  0.078 |  4.644 |  1.614 | 8
  0.000 |  0.022 |  0.085 |  0.000 |  0.107 |  5.577 |  1.168 | 16
  0.000 |  0.027 |  0.055 |  0.000 |  0.083 |  4.593 |  1.512 | 32
  0.000 |  0.028 |  0.062 |  0.000 |  0.090 |  4.433 |  1.392 | 64
  0.000 |  0.033 |  0.064 |  0.000 |  0.098 |  3.761 |  1.284 | 128
  0.000 |  0.036 |  0.083 |  0.000 |  0.119 |  3.513 |  1.057 | 256
  0.000 |  0.050 |  0.127 |  0.000 |  0.178 |  2.509 |  0.706 | 512
  0.000 |  0.062 |  0.206 |  0.000 |  0.268 |  2.029 |  0.467 | 1024
  0.000 |  0.079 |  0.480 |  0.000 |  0.560 |  1.578 |  0.223 | 2048
  0.000 |  0.124 |  0.800 |  0.000 |  0.925 |  1.006 |  0.135 | 4096

write | region_spacing | nc-nc | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.029 |  0.053 |  0.000 |  0.083 |  4.292 |  1.519 | 8
  0.000 |  0.029 |  0.038 |  0.000 |  0.067 |  4.297 |  1.869 | 16
  0.000 |  0.028 |  0.031 |  0.000 |  0.059 |  4.423 |  2.116 | 32
  0.000 |  0.028 |  0.052 |  0.000 |  0.079 |  4.529 |  1.579 | 64
  0.000 |  0.038 |  0.055 |  0.000 |  0.093 |  3.264 |  1.342 | 128
  0.001 |  0.030 |  0.081 |  0.000 |  0.112 |  4.113 |  1.123 | 256
  0.000 |  0.046 |  0.122 |  0.000 |  0.168 |  2.733 |  0.747 | 512
  0.000 |  0.059 |  0.209 |  0.000 |  0.268 |  2.136 |  0.467 | 1024
  0.000 |  0.090 |  0.402 |  0.000 |  0.492 |  1.394 |  0.254 | 2048
  0.000 |  0.078 |  0.764 |  0.000 |  0.842 |  1.606 |  0.149 | 4096

read | region_count | c-c | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.025 |  0.025 |  0.000 |  0.050 |  2.499 |  1.261 | 2048
  0.000 |  0.019 |  0.001 |  0.000 |  0.020 |  6.548 |  6.320 | 4096
  0.000 |  0.010 |  0.001 |  0.000 |  0.012 | 24.455 | 21.539 | 8192
  0.000 |  0.001 |  0.007 |  0.000 |  0.008 |355.149 | 62.376 | 16384
  0.000 |  0.015 |  0.003 |  0.000 |  0.018 | 67.331 | 56.856 | 32768
  0.000 |  0.020 |  0.006 |  0.000 |  0.026 |102.459 | 78.339 | 65536
  0.013 |  0.043 |  0.036 |  0.000 |  0.092 | 92.627 | 50.602 | 131072
  0.001 |  0.132 |  0.000 |  0.000 |  0.133 | 60.477 | 60.460 | 262144
  0.000 |  0.239 |  0.033 |  0.000 |  0.272 | 66.844 | 58.832 | 524288
  0.000 |  0.505 |  0.000 |  0.000 |  0.505 | 63.415 | 63.411 | 1048576

read | region_count | c-c | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.045 |  0.000 |  0.000 |  0.046 |  1.380 |  1.380 | 2048
  0.000 |  0.019 |  0.000 |  0.000 |  0.019 |  6.738 |  6.734 | 4096
  0.000 |  0.001 |  0.001 |  0.000 |  0.002 |290.384 |162.143 | 8192
  0.000 |  0.003 |  0.000 |  0.000 |  0.003 |169.494 |169.098 | 16384
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 |191.862 |191.311 | 32768
  0.000 |  0.027 |  0.000 |  0.000 |  0.027 | 74.407 | 74.347 | 65536
  0.000 |  0.064 |  0.000 |  0.000 |  0.064 | 62.745 | 62.712 | 131072
  0.000 |  0.128 |  0.000 |  0.000 |  0.129 | 62.311 | 62.293 | 262144
  0.000 |  0.087 |  0.178 |  0.000 |  0.266 |183.280 | 60.345 | 524288
  0.000 |  0.426 |  0.155 |  0.000 |  0.582 | 75.165 | 55.061 | 1048576

read | region_count | c-c | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.016 |  0.000 |  0.000 |  0.016 |  3.949 |  3.948 | 2048
  0.000 |  0.017 |  0.000 |  0.000 |  0.017 |  7.453 |  7.452 | 4096
  0.000 |  0.012 |  0.000 |  0.000 |  0.012 | 21.722 | 21.719 | 8192
  0.000 |  0.003 |  0.000 |  0.000 |  0.003 |171.294 |170.764 | 16384
  0.000 |  0.014 |  0.000 |  0.000 |  0.014 | 72.848 | 72.817 | 32768
  0.000 |  0.034 |  0.000 |  0.000 |  0.034 | 58.922 | 58.884 | 65536
  0.000 |  0.063 |  0.001 |  0.003 |  0.067 | 63.648 | 62.640 | 131072
  0.000 |  0.133 |  0.000 |  0.000 |  0.134 | 60.083 | 60.078 | 262144
  0.000 |  0.238 |  0.000 |  0.000 |  0.238 | 67.321 | 67.289 | 524288
  0.000 |  0.524 |  0.000 |  0.000 |  0.525 | 61.081 | 61.048 | 1048576

read | region_count | c-c | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.011 |  0.000 |  0.000 |  0.012 |  5.497 |  5.494 | 2048
  0.000 |  0.014 |  0.000 |  0.000 |  0.014 |  8.766 |  8.763 | 4096
  0.000 |  0.002 |  0.000 |  0.000 |  0.002 |131.582 |131.170 | 8192
  0.000 |  0.003 |  0.000 |  0.000 |  0.003 |157.871 |157.078 | 16384
  0.000 |  0.006 |  0.000 |  0.000 |  0.006 |176.276 |176.091 | 32768
  0.000 |  0.016 |  0.000 |  0.000 |  0.016 |127.193 |126.726 | 65536
  0.000 |  0.041 |  0.000 |  0.000 |  0.042 | 97.120 | 97.061 | 131072
  0.000 |  0.123 |  0.000 |  0.000 |  0.124 | 64.886 | 64.880 | 262144
  0.000 |  0.243 |  0.000 |  0.000 |  0.243 | 65.921 | 65.896 | 524288
  0.000 |  0.533 |  0.000 |  0.000 |  0.534 | 60.045 | 60.008 | 1048576

read | region_count | c-c | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.044 |  0.000 |  0.000 |  0.044 |  1.421 |  1.420 | 2048
  0.000 |  0.015 |  0.001 |  0.000 |  0.016 |  8.229 |  7.874 | 4096
  0.000 |  0.010 |  0.001 |  0.000 |  0.012 | 24.699 | 21.785 | 8192
  0.000 |  0.002 |  0.001 |  0.000 |  0.003 |289.183 |161.805 | 16384
  0.000 |  0.003 |  0.005 |  0.000 |  0.008 |360.243 |127.813 | 32768
  0.000 |  0.036 |  0.000 |  0.000 |  0.036 | 56.245 | 56.213 | 65536
  0.000 |  0.056 |  0.018 |  0.000 |  0.075 | 70.869 | 53.832 | 131072
  0.000 |  0.060 |  0.078 |  0.000 |  0.139 |133.360 | 57.823 | 262144
  0.000 |  0.052 |  0.228 |  0.000 |  0.281 |305.314 | 57.124 | 524288
  0.000 |  0.531 |  0.060 |  0.000 |  0.591 | 60.318 | 54.208 | 1048576

read | region_count | c-c | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.046 |  0.000 |  0.000 |  0.046 |  1.358 |  1.358 | 2048
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |141.394 |140.748 | 4096
  0.000 |  0.019 |  0.000 |  0.000 |  0.019 | 13.015 | 13.009 | 8192
  0.000 |  0.002 |  0.001 |  0.000 |  0.003 |291.393 |160.874 | 16384
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |196.925 |196.574 | 32768
  0.000 |  0.033 |  0.000 |  0.000 |  0.034 | 59.841 | 59.805 | 65536
  0.000 |  0.062 |  0.000 |  0.000 |  0.063 | 64.258 | 64.226 | 131072
  0.000 |  0.128 |  0.000 |  0.000 |  0.129 | 62.278 | 62.246 | 262144
  0.000 |  0.137 |  0.097 |  0.000 |  0.235 |116.460 | 68.230 | 524288
  0.010 |  0.370 |  0.231 |  0.000 |  0.612 | 86.490 | 53.203 | 1048576

read | region_count | nc-nc | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.006 |  0.015 |  0.000 |  0.000 |  0.022 |  4.051 |  4.039 | 2048
  0.008 |  0.035 |  0.000 |  0.000 |  0.043 |  3.556 |  3.553 | 4096
  0.003 |  0.067 |  0.000 |  0.000 |  0.070 |  3.741 |  3.739 | 8192
  0.010 |  0.132 |  0.000 |  0.000 |  0.142 |  3.802 |  3.801 | 16384
  0.012 |  0.251 |  0.000 |  0.000 |  0.263 |  3.978 |  3.977 | 32768
  0.006 |  0.656 |  0.000 |  0.000 |  0.662 |  3.050 |  3.050 | 65536
  0.015 |  1.265 |  0.000 |  0.000 |  1.279 |  3.162 |  3.162 | 131072
  0.013 |  2.386 |  0.000 |  0.000 |  2.399 |  3.354 |  3.354 | 262144
  0.032 |  4.806 |  0.000 |  0.000 |  4.838 |  3.329 |  3.329 | 524288
  0.056 |  8.382 |  0.000 |  0.000 |  8.438 |  3.818 |  3.818 | 1048576

read | region_count | nc-nc | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.017 |  0.025 |  0.003 |  0.000 |  0.045 |  2.536 |  2.238 | 2048
  0.007 |  0.018 |  0.000 |  0.000 |  0.026 |  6.965 |  6.897 | 4096
  0.004 |  0.067 |  0.000 |  0.000 |  0.071 |  3.746 |  3.744 | 8192
  0.010 |  0.124 |  0.000 |  0.000 |  0.134 |  4.047 |  4.032 | 16384
  0.005 |  0.258 |  0.000 |  0.000 |  0.263 |  3.876 |  3.875 | 32768
  0.010 |  0.481 |  0.000 |  0.000 |  0.492 |  4.156 |  4.156 | 65536
  0.011 |  1.084 |  0.000 |  0.000 |  1.095 |  3.692 |  3.692 | 131072
  0.022 |  2.116 |  0.000 |  0.000 |  2.138 |  3.780 |  3.780 | 262144
  0.024 |  4.065 |  0.000 |  0.000 |  4.090 |  3.936 |  3.936 | 524288
  0.055 |  8.127 |  0.000 |  0.000 |  8.182 |  3.938 |  3.938 | 1048576

read | region_count | nc-nc | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.001 |  0.006 |  0.000 |  0.000 |  0.006 | 10.988 | 10.924 | 2048
  0.015 |  0.054 |  0.000 |  0.000 |  0.069 |  2.319 |  2.317 | 4096
  0.003 |  0.085 |  0.000 |  0.000 |  0.089 |  2.930 |  2.920 | 8192
  0.025 |  0.169 |  0.001 |  0.000 |  0.195 |  2.954 |  2.940 | 16384
  0.009 |  0.351 |  0.002 |  0.000 |  0.363 |  2.846 |  2.830 | 32768
  0.009 |  0.636 |  0.005 |  0.000 |  0.651 |  3.145 |  3.118 | 65536
  0.020 |  1.247 |  0.001 |  0.000 |  1.268 |  3.208 |  3.206 | 131072
  0.033 |  2.334 |  0.000 |  0.000 |  2.368 |  3.427 |  3.427 | 262144
  0.031 |  4.871 |  0.001 |  0.000 |  4.904 |  3.284 |  3.284 | 524288
  0.058 |  9.787 |  0.003 |  0.000 |  9.848 |  3.270 |  3.269 | 1048576

read | region_count | nc-nc | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.016 |  0.023 |  0.000 |  0.000 |  0.039 |  2.662 |  2.660 | 2048
  0.008 |  0.048 |  0.000 |  0.000 |  0.056 |  2.622 |  2.620 | 4096
  0.008 |  0.076 |  0.000 |  0.000 |  0.085 |  3.270 |  3.258 | 8192
  0.009 |  0.169 |  0.000 |  0.000 |  0.178 |  2.956 |  2.956 | 16384
  0.011 |  0.335 |  0.000 |  0.000 |  0.347 |  2.981 |  2.980 | 32768
  0.008 |  0.686 |  0.002 |  0.000 |  0.696 |  2.914 |  2.906 | 65536
  0.014 |  1.264 |  0.000 |  0.000 |  1.279 |  3.165 |  3.164 | 131072
  0.020 |  2.370 |  0.001 |  0.000 |  2.391 |  3.376 |  3.375 | 262144
  0.031 |  4.832 |  0.000 |  0.000 |  4.863 |  3.312 |  3.312 | 524288
  0.054 |  9.518 |  0.053 |  0.000 |  9.625 |  3.362 |  3.343 | 1048576

read | region_count | nc-nc | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.016 |  0.019 |  0.006 |  0.000 |  0.040 |  3.354 |  2.537 | 2048
  0.007 |  0.036 |  0.000 |  0.000 |  0.043 |  3.506 |  3.502 | 4096
  0.005 |  0.073 |  0.000 |  0.000 |  0.078 |  3.433 |  3.432 | 8192
  0.004 |  0.117 |  0.000 |  0.000 |  0.121 |  4.282 |  4.281 | 16384
  0.004 |  0.240 |  0.000 |  0.000 |  0.244 |  4.170 |  4.169 | 32768
  0.005 |  0.463 |  0.000 |  0.000 |  0.468 |  4.324 |  4.323 | 65536
  0.011 |  0.916 |  0.000 |  0.000 |  0.927 |  4.368 |  4.368 | 131072
  0.016 |  0.652 |  0.000 |  0.000 |  0.668 | 12.269 | 12.268 | 262144
  0.019 |  0.682 |  0.000 |  0.000 |  0.701 | 23.467 | 23.466 | 524288
  0.036 |  1.196 |  0.017 |  0.000 |  1.250 | 26.756 | 26.373 | 1048576

read | region_count | nc-nc | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_count
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 | 26.283 | 26.009 | 2048
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.462 | 26.389 | 4096
  0.001 |  0.009 |  0.000 |  0.000 |  0.010 | 26.972 | 26.093 | 8192
  0.001 |  0.019 |  0.000 |  0.000 |  0.020 | 26.356 | 26.199 | 16384
  0.001 |  0.037 |  0.000 |  0.000 |  0.038 | 27.070 | 27.053 | 32768
  0.002 |  0.075 |  0.000 |  0.000 |  0.078 | 26.634 | 26.626 | 65536
  0.006 |  0.148 |  0.004 |  0.000 |  0.157 | 27.044 | 26.339 | 131072
  0.010 |  0.295 |  0.009 |  0.000 |  0.314 | 27.121 | 26.325 | 262144
  0.020 |  0.598 |  0.004 |  0.000 |  0.623 | 26.763 | 26.567 | 524288
  0.037 |  1.203 |  0.014 |  0.000 |  1.254 | 26.590 | 26.291 | 1048576

read | region_size | c-c | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |2449.944 |2080.508 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3578.758 |3206.654 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3792.318 |3548.481 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |3355.443 |3266.592 | 64
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |2341.878 |2317.295 | 128
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 |1755.123 |1633.932 | 256
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |1747.809 |1672.203 | 512
  0.000 |  0.009 |  0.001 |  0.000 |  0.010 |1858.765 |1747.126 | 1024
  0.000 |  0.017 |  0.001 |  0.000 |  0.019 |1903.960 |1779.250 | 2048
  0.000 |  0.035 |  0.001 |  0.000 |  0.037 |1822.954 |1768.256 | 4096

read | region_size | c-c | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |2716.518 |2545.087 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3518.711 |3426.719 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3792.318 |3705.216 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |3701.945 |3412.778 | 64
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |2039.039 |2014.072 | 128
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 |1810.035 |1665.232 | 256
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 |1793.300 |1677.889 | 512
  0.000 |  0.009 |  0.001 |  0.000 |  0.010 |1869.588 |1721.711 | 1024
  0.000 |  0.017 |  0.001 |  0.000 |  0.019 |1902.044 |1762.613 | 2048
  0.000 |  0.034 |  0.002 |  0.000 |  0.036 |1897.701 |1778.272 | 4096

read | region_size | c-c | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |405.796 |401.753 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1255.780 |1243.862 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |1219.274 |1190.211 | 32
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |1177.844 |1172.248 | 64
  0.000 |  0.003 |  0.000 |  0.000 |  0.004 |677.047 |612.218 | 128
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 |740.193 |715.050 | 256
  0.001 |  0.009 |  0.000 |  0.000 |  0.010 |849.888 |826.362 | 512
  0.000 |  0.018 |  0.000 |  0.000 |  0.019 |876.427 |863.092 | 1024
  0.000 |  0.036 |  0.001 |  0.000 |  0.037 |897.898 |883.197 | 2048
  0.000 |  0.080 |  0.001 |  0.000 |  0.082 |801.714 |787.150 | 4096

read | region_size | c-c | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |589.750 |581.251 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1184.832 |1174.217 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |1201.806 |1174.217 | 32
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |1184.832 |1173.889 | 64
  0.000 |  0.003 |  0.000 |  0.000 |  0.004 |688.042 |605.894 | 128
  0.000 |  0.006 |  0.000 |  0.000 |  0.006 |726.192 |708.198 | 256
  0.000 |  0.009 |  0.000 |  0.000 |  0.009 |902.632 |900.597 | 512
  0.000 |  0.018 |  0.000 |  0.000 |  0.019 |883.488 |866.413 | 1024
  0.000 |  0.036 |  0.000 |  0.000 |  0.037 |892.209 |886.552 | 2048
  0.000 |  0.080 |  0.001 |  0.000 |  0.082 |799.870 |786.278 | 4096

read | region_size | c-c | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |2496.610 |2351.067 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3971.879 |3679.214 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4002.198 |3847.985 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |3066.012 |3002.365 | 64
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |2166.479 |2148.171 | 128
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 |1797.816 |1569.284 | 256
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 |1827.285 |1656.273 | 512
  0.001 |  0.009 |  0.001 |  0.000 |  0.010 |1879.274 |1759.033 | 1024
  0.001 |  0.017 |  0.001 |  0.000 |  0.018 |1917.424 |1793.923 | 2048
  0.000 |  0.034 |  0.002 |  0.000 |  0.036 |1895.851 |1779.663 | 4096

read | region_size | c-c | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |2833.989 |2496.610 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3679.214 |3578.758 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3876.436 |3758.337 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.001 |3688.922 |3390.707 | 64
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |2833.032 |1818.077 | 128
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 |1797.045 |1660.453 | 256
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |1708.300 |1692.361 | 512
  0.000 |  0.008 |  0.001 |  0.000 |  0.010 |1890.550 |1740.827 | 1024
  0.000 |  0.017 |  0.001 |  0.000 |  0.018 |1913.651 |1792.414 | 2048
  0.000 |  0.033 |  0.003 |  0.000 |  0.036 |1923.028 |1779.958 | 4096

read | region_size | nc-nc | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 27.203 | 26.212 | 8
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 51.718 | 51.578 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |105.639 |105.215 | 32
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |216.123 |215.568 | 64
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 |389.932 |382.849 | 128
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 |750.323 |691.930 | 256
  0.001 |  0.007 |  0.001 |  0.000 |  0.008 |1187.473 |1088.899 | 512
  0.000 |  0.012 |  0.000 |  0.000 |  0.012 |1373.999 |1370.940 | 1024
  0.001 |  0.019 |  0.003 |  0.000 |  0.022 |1698.529 |1492.817 | 2048
  0.001 |  0.036 |  0.003 |  0.000 |  0.040 |1769.130 |1637.840 | 4096

read | region_size | nc-nc | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 24.974 | 24.534 | 8
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 47.314 | 46.469 | 16
  0.000 |  0.006 |  0.000 |  0.000 |  0.007 | 79.949 | 79.785 | 32
  0.000 |  0.007 |  0.000 |  0.000 |  0.008 |134.592 |130.925 | 64
  0.001 |  0.010 |  0.001 |  0.000 |  0.011 |209.820 |199.302 | 128
  0.001 |  0.015 |  0.001 |  0.000 |  0.016 |270.252 |257.782 | 256
  0.001 |  0.024 |  0.000 |  0.000 |  0.025 |329.055 |327.479 | 512
  0.000 |  0.043 |  0.000 |  0.000 |  0.044 |368.275 |366.537 | 1024
  0.000 |  0.083 |  0.002 |  0.000 |  0.086 |383.367 |373.968 | 2048
  0.000 |  0.159 |  0.002 |  0.000 |  0.161 |403.258 |397.903 | 4096

read | region_size | nc-nc | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 25.243 | 25.147 | 8
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 59.088 | 58.935 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 98.310 | 97.275 | 32
  0.000 |  0.007 |  0.000 |  0.000 |  0.008 |142.083 |141.982 | 64
  0.000 |  0.009 |  0.000 |  0.000 |  0.010 |214.405 |204.083 | 128
  0.000 |  0.013 |  0.002 |  0.000 |  0.015 |313.851 |278.497 | 256
  0.000 |  0.021 |  0.001 |  0.000 |  0.023 |372.475 |358.940 | 512
  0.000 |  0.036 |  0.005 |  0.000 |  0.041 |445.370 |394.545 | 1024
  0.000 |  0.067 |  0.010 |  0.000 |  0.077 |480.198 |417.988 | 2048
  0.000 |  0.152 |  0.000 |  0.000 |  0.153 |420.646 |420.627 | 4096

read | region_size | nc-nc | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.001 |  0.005 |  0.000 |  0.000 |  0.005 | 25.784 | 25.757 | 8
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 59.936 | 58.206 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 |100.544 | 98.523 | 32
  0.000 |  0.007 |  0.000 |  0.000 |  0.008 |141.767 |140.867 | 64
  0.000 |  0.009 |  0.001 |  0.000 |  0.010 |218.175 |205.131 | 128
  0.001 |  0.014 |  0.000 |  0.000 |  0.015 |285.613 |285.307 | 256
  0.001 |  0.020 |  0.002 |  0.000 |  0.023 |401.470 |358.618 | 512
  0.000 |  0.036 |  0.005 |  0.000 |  0.041 |450.326 |395.014 | 1024
  0.001 |  0.067 |  0.010 |  0.000 |  0.077 |479.537 |418.656 | 2048
  0.000 |  0.152 |  0.000 |  0.000 |  0.153 |420.107 |420.077 | 4096

read | region_size | nc-nc | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.952 | 26.179 | 8
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 50.887 | 50.764 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |107.878 |107.480 | 32
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 |222.427 |216.547 | 64
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 |399.115 |396.343 | 128
  0.001 |  0.006 |  0.000 |  0.000 |  0.006 |705.221 |703.506 | 256
  0.000 |  0.007 |  0.000 |  0.000 |  0.008 |1136.552 |1071.821 | 512
  0.001 |  0.010 |  0.001 |  0.000 |  0.013 |1551.005 |1393.370 | 1024
  0.001 |  0.019 |  0.002 |  0.000 |  0.021 |1680.852 |1542.839 | 2048
  0.001 |  0.039 |  0.001 |  0.000 |  0.040 |1651.148 |1620.958 | 4096

read | region_size | nc-nc | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_size
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 27.569 | 27.496 | 8
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 54.395 | 52.600 | 16
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 |112.944 |107.227 | 32
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 |231.755 |222.616 | 64
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 |405.364 |404.290 | 128
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 |760.010 |699.663 | 256
  0.001 |  0.007 |  0.000 |  0.000 |  0.008 |1097.088 |1093.334 | 512
  0.001 |  0.012 |  0.000 |  0.000 |  0.012 |1388.296 |1385.173 | 1024
  0.000 |  0.021 |  0.000 |  0.000 |  0.021 |1546.715 |1544.472 | 2048
  0.000 |  0.036 |  0.003 |  0.000 |  0.040 |1760.082 |1628.579 | 4096

read | region_spacing | c-c | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |2716.518 |2449.944 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3692.169 |3472.106 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3120.762 |2962.079 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3048.186 |2404.991 | 64
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3912.597 |3566.585 | 128
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3771.856 |3472.106 | 256
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3472.106 |3196.878 | 512
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3799.188 | 1024
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3566.585 | 2048
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4032.985 |3666.350 | 4096

read | region_spacing | c-c | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3799.188 |3382.503 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3276.800 |3048.186 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3799.188 |3591.014 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3883.615 | 64
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4297.443 |3883.615 | 128
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4443.119 |3883.615 | 256
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3666.350 | 512
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4332.959 |4032.985 | 1024
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3566.585 | 2048
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4332.959 |3912.597 | 4096

read | region_spacing | c-c | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |722.160 |714.289 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1291.350 |1251.284 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1358.259 |1330.680 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1436.405 |1387.005 | 64
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1067.796 |1024.000 | 128
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1436.405 |1390.684 | 256
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1452.321 |1416.995 | 512
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1405.598 |1358.259 | 1024
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1314.005 |1275.640 | 2048
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1489.455 |1436.405 | 4096

read | region_spacing | c-c | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1436.405 |1372.482 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1452.321 |1405.598 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1468.594 |1436.405 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1452.321 |1420.835 | 64
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1436.405 |1387.005 | 128
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1224.972 |1202.495 | 256
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1420.835 |1390.684 | 512
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1456.356 |1420.835 | 1024
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1401.840 |1372.482 | 2048
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |1468.594 |1436.405 | 4096

read | region_spacing | c-c | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3771.856 |3566.585 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3771.856 |3666.350 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3912.597 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |3799.188 |1524.093 | 64
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4194.304 |3692.169 | 128
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4332.959 |3912.597 | 256
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4443.119 |3912.597 | 512
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3566.585 | 1024
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3799.188 | 2048
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4481.094 |3912.597 | 4096

read | region_spacing | c-c | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4332.959 |4032.985 | 8
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3472.106 | 16
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4161.016 |3799.188 | 32
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4297.443 |3883.615 | 64
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4809.982 |4161.016 | 128
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4443.119 |3883.615 | 256
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4297.443 |3771.856 | 512
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4639.717 |4032.985 | 1024
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4639.717 |4032.985 | 2048
  0.000 |  0.000 |  0.000 |  0.000 |  0.000 |4332.959 |3799.188 | 4096

read | region_spacing | nc-nc | posix
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 27.772 | 27.516 | 8
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 27.581 | 27.545 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.394 | 26.305 | 32
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.365 | 26.281 | 64
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.545 | 26.228 | 128
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 25.933 | 25.594 | 256
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 26.245 | 24.955 | 512
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 24.357 | 23.938 | 1024
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 | 23.837 | 23.723 | 2048
  0.001 |  0.005 |  0.000 |  0.000 |  0.005 | 26.289 | 25.742 | 4096

read | region_spacing | nc-nc | datasieve
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |231.474 |198.444 | 8
  0.000 |  0.001 |  0.000 |  0.000 |  0.001 |191.696 |189.959 | 16
  0.000 |  0.001 |  0.000 |  0.000 |  0.002 | 85.738 | 77.397 | 32
  0.000 |  0.003 |  0.000 |  0.000 |  0.003 | 42.159 | 40.089 | 64
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 25.020 | 24.390 | 128
  0.001 |  0.009 |  0.000 |  0.000 |  0.009 | 14.353 | 14.320 | 256
  0.000 |  0.016 |  0.000 |  0.000 |  0.017 |  7.840 |  7.774 | 512
  0.000 |  0.031 |  0.000 |  0.000 |  0.031 |  4.086 |  4.080 | 1024
  0.000 |  0.066 |  0.000 |  0.000 |  0.067 |  1.887 |  1.886 | 2048
  0.000 |  0.117 |  0.000 |  0.000 |  0.118 |  1.066 |  1.066 | 4096

read | region_spacing | nc-nc | two_phase
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 | 65.479 | 59.409 | 8
  0.000 |  0.002 |  0.000 |  0.000 |  0.002 | 59.748 | 58.929 | 16
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 | 50.080 | 49.942 | 32
  0.000 |  0.003 |  0.000 |  0.000 |  0.004 | 40.191 | 38.519 | 64
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 31.010 | 30.362 | 128
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 24.136 | 24.103 | 256
  0.001 |  0.007 |  0.000 |  0.000 |  0.008 | 16.970 | 16.678 | 512
  0.001 |  0.012 |  0.000 |  0.000 |  0.013 | 10.522 | 10.410 | 1024
  0.000 |  0.021 |  0.000 |  0.000 |  0.022 |  5.972 |  5.953 | 2048
  0.000 |  0.060 |  0.000 |  0.000 |  0.061 |  2.079 |  2.079 | 4096

read | region_spacing | nc-nc | two_phase_PFR
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.002 |  0.000 |  0.000 |  0.002 | 60.794 | 60.618 | 8
  0.000 |  0.002 |  0.000 |  0.000 |  0.002 | 59.864 | 59.578 | 16
  0.000 |  0.002 |  0.000 |  0.000 |  0.003 | 51.105 | 47.784 | 32
  0.000 |  0.003 |  0.000 |  0.000 |  0.004 | 38.556 | 38.180 | 64
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 31.361 | 30.274 | 128
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 | 23.733 | 23.209 | 256
  0.000 |  0.008 |  0.000 |  0.000 |  0.008 | 16.366 | 16.146 | 512
  0.000 |  0.012 |  0.000 |  0.000 |  0.013 | 10.319 | 10.292 | 1024
  0.000 |  0.023 |  0.000 |  0.000 |  0.024 |  5.433 |  5.387 | 2048
  0.000 |  0.039 |  0.000 |  0.000 |  0.040 |  3.200 |  3.199 | 4096

read | region_spacing | nc-nc | list
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 29.614 | 29.468 | 8
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 28.377 | 28.286 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 25.418 | 25.319 | 32
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.014 | 25.896 | 64
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 25.111 | 25.040 | 128
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 25.987 | 25.859 | 256
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 25.120 | 25.065 | 512
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 24.558 | 24.429 | 1024
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 | 24.155 | 23.997 | 2048
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 | 25.977 | 25.283 | 4096

read | region_spacing | nc-nc | datatype
----------------time (seconds)--------------|-bandwidth (MB/s)|---parameter---
  open  |   io   |  sync  | close  | total  |   IO   |  IOsyn | region_spacing
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 27.545 | 27.509 | 8
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 28.755 | 28.656 | 16
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 25.573 | 25.473 | 32
  0.000 |  0.004 |  0.000 |  0.000 |  0.005 | 28.300 | 27.821 | 64
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 27.263 | 25.594 | 128
  0.000 |  0.005 |  0.000 |  0.000 |  0.005 | 26.618 | 26.556 | 256
  0.001 |  0.005 |  0.000 |  0.000 |  0.005 | 25.484 | 25.350 | 512
  0.000 |  0.005 |  0.000 |  0.000 |  0.006 | 24.510 | 24.381 | 1024
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 | 24.323 | 24.248 | 2048
  0.001 |  0.005 |  0.000 |  0.000 |  0.006 | 25.422 | 25.356 | 4096

**All (720) test(s) passed without any errors!**

Output has been created using version 3.0.1.