logo
Virtual Institute
for
I/O

User Tools

Site Tools

You are here: Virtual Institute for I/O » Tools » Benchmarks » MDTest

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
tools:benchmarks:mdtest [2016/12/02 11:17] – created kunkeltools:benchmarks:mdtest [2021/09/03 11:22] (current) – [MDTest] kunkel
Line 1: Line 1:
-====== IOR ======+====== MDTest ======
 ---- dataentry benchmark ---- ---- dataentry benchmark ----
-name           : mdtest +name           : MDTest #  
-layers_ttags   : POSIX, S3 +layers_ttags   : POSIX, S3 #  
-webpage_url    : https://github.com/llnl/mdtest +features_ttags : metadata #  
-license_ttag   : GPL +parallel_ttags : MPI #  
-type_ttags     : synthetic+type_ttags     : synthetic #  
 +license_ttag   : GPL #  
 +webpage_url    : https://github.com/hpc/ior # 
 ---- ----
  
-mdtest is a synthetic MPI parallel benchmark for testing the metadata performance of filesystems.+ 
 +MDTest is a synthetic MPI parallel benchmark for testing the metadata performance of filesystems.
 Each thread operates on its own working set.  Each thread operates on its own working set. 
  
Line 22: Line 25:
 Here are a few important command line arguments: Here are a few important command line arguments:
   * ''-d'' Target directory to run the test on   * ''-d'' Target directory to run the test on
-  * ''-b'' Branching factor, how many directories to create  +  * ''-i'' Iterations, the benchmark is repeated this number of times to investigate standard deviation 
-  * ''-i'' Iterations+  * ''-z'' Depth of the hierarchical directory structure 
 +  * ''-b'' Branching factor, how many directories to create, used in conjunction with -z
   * ''-I'' Number of files to create per directory   * ''-I'' Number of files to create per directory
 +  * ''-w'' Write this number of bytes to each file after it is created
 +  * ''-e'' Read this number of bytes from each file in the read phase
  
 +To print the internal operations of the benchmark for rank 0 add the option ''-V3''.
 ===== Example Output ===== ===== Example Output =====
-For example runnning ''mpiexec -n 2 ./mdtest -d /tmp/ -b 10 -i 1 -I 100'' (using mdtest version 1.9.3) results in the following output +For example runnning ''mpiexec -n 2 ./mdtest -d /tmp/ -b 10 -z 2 -i 1 -I 100'' (using mdtest version 1.9.3) results in the following output 
 <code> <code>
--- started at 12/02/2016 11:15:22 --+-- started at 12/02/2016 11:24:25 --
  
 mdtest-1.9.3 was launched with 2 total task(s) on 1 node(s) mdtest-1.9.3 was launched with 2 total task(s) on 1 node(s)
-Command line used: ./mdtest -d /tmp/ -b 10 -i 1 -I 100+Command line used: ./mdtest -d /tmp/ -b 10 -z 2 -i 1 -I 100
 Path: /tmp Path: /tmp
 FS: 15.6 GiB   Used FS: 0.0%   Inodes: 3.9 Mi   Used Inodes: 0.0% FS: 15.6 GiB   Used FS: 0.0%   Inodes: 3.9 Mi   Used Inodes: 0.0%
  
-2 tasks, 200 files/directories+2 tasks, 22200 files/directories
  
 SUMMARY: (of 1 iterations) SUMMARY: (of 1 iterations)
    Operation                      Max            Min           Mean        Std Dev    Operation                      Max            Min           Mean        Std Dev
    ---------                      ---            ---           ----        -------    ---------                      ---            ---           ----        -------
-   Directory creation:     452704.155     452704.155     452704.155          0.000 +   Directory creation:     980917.027     980917.027     980917.027          0.000 
-   Directory stat    :     900065.236     900065.236     900065.236          0.000 +   Directory stat    :    1509378.324    1509378.324    1509378.324          0.000 
-   Directory removal :     502612.822     502612.822     502612.822          0.000 +   Directory removal :    1065726.028    1065726.028    1065726.028          0.000 
-   File creation     :     353353.328     353353.328     353353.328          0.000 +   File creation     :     845203.634     845203.634     845203.634          0.000 
-   File stat         :    1184831.638    1184831.638    1184831.638          0.000 +   File stat         :    1563488.352    1563488.352    1563488.352          0.000 
-   File read         :     783982.056     783982.056     783982.056          0.000 +   File read         :    1064592.847    1064592.847    1064592.847          0.000 
-   File removal      :     716362.767     716362.767     716362.767          0.000 +   File removal      :    1214044.210    1214044.210    1214044.210          0.000 
-   Tree creation     :     199728.762     199728.762     199728.762          0.000 +   Tree creation     :     734333.981     734333.981     734333.981          0.000 
-   Tree removal      :     246723.765     246723.765     246723.765          0.000+   Tree removal      :     860568.843     860568.843     860568.843          0.000 
 +    
 +-- finished at 12/02/2016 11:24:25 -- 
 +</code>
  
--- finished at 12/02/2016 11:15:22 --+ 
 +==== Results on an HDD ==== 
 + 
 +  * CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz 
 +  * HDD:  HGST HTS721010A9 A3P0 
 + 
 +$ mpiexec -n 8 ./mdtest -z 1 -L -u -n -N 1 -b 10 -d /hdd/local/benchmark/ -I 1000 -w 3900 -e 3900  
 + 
 +<code> 
 +-- started at 12/02/2016 15:35:49 -- 
 + 
 +mdtest-1.9.3 was launched with 8 total task(s) on 1 node(s) 
 +Command line used: ./mdtest -z 1 -L -u -n -N 1 -b 10 -d /hdd/local/benchmark/ -I 1000 -w 3900 -e 3900 
 +Path: /hdd/local/benchmark 
 +FS: 916.8 GiB   Used FS: 17.1%   Inodes: 58.2 Mi   Used Inodes: 0.8% 
 + 
 +8 tasks, 88000 files/directories 
 + 
 +SUMMARY: (of 1 iterations) 
 +   Operation                      Max            Min           Mean        Std Dev 
 +   ---------                      ---            ---           ----        ------- 
 +   Directory creation:       6927.308       6927.308       6927.308          0.000 
 +   Directory stat    :    4238519.464    4238519.464    4238519.464          0.000 
 +   Directory removal :     730975.466     730975.466     730975.466          0.000 
 +   File creation     :      48177.549      48177.549      48177.549          0.000 
 +   File stat         :    3944079.074    3944079.074    3944079.074          0.000 
 +   File read         :    1853445.039    1853445.039    1853445.039          0.000 
 +   File removal      :     977343.745     977343.745     977343.745          0.000 
 +   Tree creation     :      98373.868      98373.868      98373.868          0.000 
 +   Tree removal      :        569.562        569.562        569.562          0.000 
 + 
 +-- finished at 12/02/2016 15:36:04 --
 </code> </code>
  
 +Here is a longer run:
 +$ mpiexec -n 8 ./mdtest -z 1 -L -b 10 -d /hdd/local/benchmark/ -I 10000 -w 3900 -e 3900
 +<code>
 +-- started at 12/02/2016 15:55:59 --
 +
 +mdtest-1.9.3 was launched with 8 total task(s) on 1 node(s)
 +Command line used: ./mdtest -z 1 -L -b 10 -d /hdd/local/benchmark/ -I 10000 -w 3900 -e 3900
 +Path: /hdd/local/benchmark
 +FS: 916.8 GiB   Used FS: 17.1%   Inodes: 58.2 Mi   Used Inodes: 0.8%
 +
 +8 tasks, 880000 files/directories
 +                                                                                         
 +mpiexec -n 8 ./mdtest -z 1 -L -b 10 -d /hdd/local/benchmark/ -I 30000 -w 3900 -e 3900
 +
 +SUMMARY: (of 1 iterations)
 +   Operation                      Max            Min           Mean        Std Dev
 +   ---------                      ---            ---           ----        -------
 +   Directory creation:       5027.549       5027.549       5027.549          0.000
 +   Directory stat    :    4453443.823    4453443.823    4453443.823          0.000
 +   Directory removal :       9901.782       9901.782       9901.782          0.000
 +   File creation     :      23445.425      23445.425      23445.425          0.000
 +   File stat         :    2970635.216    2970635.216    2970635.216          0.000
 +   File read         :      55775.043      55775.043      55775.043          0.000
 +   File removal      :      22110.331      22110.331      22110.331          0.000
 +   Tree creation     :      90112.000      90112.000      90112.000          0.000
 +   Tree removal      :         25.082         25.082         25.082          0.000
 +
 +-- finished at 12/02/2016 16:01:57 --
 +</code>
tools/benchmarks/mdtest.1480673855.txt.gz · Last modified: 2016/12/02 11:17 by kunkel
Impressum Privacy Driven by DokuWiki CC Attribution-Share Alike 4.0 International