====== MDTest ======
---- dataentry benchmark ----
name : MDTest #
layers_ttags : POSIX, S3 #
features_ttags : metadata #
parallel_ttags : MPI #
type_ttags : synthetic #
license_ttag : GPL #
webpage_url : https://github.com/hpc/ior #
----
MDTest is a synthetic MPI parallel benchmark for testing the metadata performance of filesystems.
Each thread operates on its own working set.
The benchmark operates in phases (excerpt):
* Creation of directories
* Creation of files
* Reading files
* Deletion of files
* Deletion of the tree
===== Usage =====
Here are a few important command line arguments:
* ''-d'' Target directory to run the test on
* ''-i'' Iterations, the benchmark is repeated this number of times to investigate standard deviation
* ''-z'' Depth of the hierarchical directory structure
* ''-b'' Branching factor, how many directories to create, used in conjunction with -z
* ''-I'' Number of files to create per directory
* ''-w'' Write this number of bytes to each file after it is created
* ''-e'' Read this number of bytes from each file in the read phase
To print the internal operations of the benchmark for rank 0 add the option ''-V3''.
===== Example Output =====
For example runnning ''mpiexec -n 2 ./mdtest -d /tmp/ -b 10 -z 2 -i 1 -I 100'' (using mdtest version 1.9.3) results in the following output
-- started at 12/02/2016 11:24:25 --
mdtest-1.9.3 was launched with 2 total task(s) on 1 node(s)
Command line used: ./mdtest -d /tmp/ -b 10 -z 2 -i 1 -I 100
Path: /tmp
FS: 15.6 GiB Used FS: 0.0% Inodes: 3.9 Mi Used Inodes: 0.0%
2 tasks, 22200 files/directories
SUMMARY: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
Directory creation: 980917.027 980917.027 980917.027 0.000
Directory stat : 1509378.324 1509378.324 1509378.324 0.000
Directory removal : 1065726.028 1065726.028 1065726.028 0.000
File creation : 845203.634 845203.634 845203.634 0.000
File stat : 1563488.352 1563488.352 1563488.352 0.000
File read : 1064592.847 1064592.847 1064592.847 0.000
File removal : 1214044.210 1214044.210 1214044.210 0.000
Tree creation : 734333.981 734333.981 734333.981 0.000
Tree removal : 860568.843 860568.843 860568.843 0.000
-- finished at 12/02/2016 11:24:25 --
==== Results on an HDD ====
* CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
* HDD: HGST HTS721010A9 A3P0
$ mpiexec -n 8 ./mdtest -z 1 -L -u -n -N 1 -b 10 -d /hdd/local/benchmark/ -I 1000 -w 3900 -e 3900
-- started at 12/02/2016 15:35:49 --
mdtest-1.9.3 was launched with 8 total task(s) on 1 node(s)
Command line used: ./mdtest -z 1 -L -u -n -N 1 -b 10 -d /hdd/local/benchmark/ -I 1000 -w 3900 -e 3900
Path: /hdd/local/benchmark
FS: 916.8 GiB Used FS: 17.1% Inodes: 58.2 Mi Used Inodes: 0.8%
8 tasks, 88000 files/directories
SUMMARY: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
Directory creation: 6927.308 6927.308 6927.308 0.000
Directory stat : 4238519.464 4238519.464 4238519.464 0.000
Directory removal : 730975.466 730975.466 730975.466 0.000
File creation : 48177.549 48177.549 48177.549 0.000
File stat : 3944079.074 3944079.074 3944079.074 0.000
File read : 1853445.039 1853445.039 1853445.039 0.000
File removal : 977343.745 977343.745 977343.745 0.000
Tree creation : 98373.868 98373.868 98373.868 0.000
Tree removal : 569.562 569.562 569.562 0.000
-- finished at 12/02/2016 15:36:04 --
Here is a longer run:
$ mpiexec -n 8 ./mdtest -z 1 -L -b 10 -d /hdd/local/benchmark/ -I 10000 -w 3900 -e 3900
-- started at 12/02/2016 15:55:59 --
mdtest-1.9.3 was launched with 8 total task(s) on 1 node(s)
Command line used: ./mdtest -z 1 -L -b 10 -d /hdd/local/benchmark/ -I 10000 -w 3900 -e 3900
Path: /hdd/local/benchmark
FS: 916.8 GiB Used FS: 17.1% Inodes: 58.2 Mi Used Inodes: 0.8%
8 tasks, 880000 files/directories
mpiexec -n 8 ./mdtest -z 1 -L -b 10 -d /hdd/local/benchmark/ -I 30000 -w 3900 -e 3900
SUMMARY: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
Directory creation: 5027.549 5027.549 5027.549 0.000
Directory stat : 4453443.823 4453443.823 4453443.823 0.000
Directory removal : 9901.782 9901.782 9901.782 0.000
File creation : 23445.425 23445.425 23445.425 0.000
File stat : 2970635.216 2970635.216 2970635.216 0.000
File read : 55775.043 55775.043 55775.043 0.000
File removal : 22110.331 22110.331 22110.331 0.000
Tree creation : 90112.000 90112.000 90112.000 0.000
Tree removal : 25.082 25.082 25.082 0.000
-- finished at 12/02/2016 16:01:57 --