====== FS: BINFS ======
===== Characteristics =====
name:BINFS
===== Description =====
The BINFS (short for Bioinformatics small) storage system was procured in late 2016 consisting of 64 x Intel P3600 2 TB SSDs distributed on 16 nodes. Running 8 metadata servers (buddy-mirrored) and 16 OSTs. The nodes are used as storage and compute nodes, therefore they are connected via Intel Omnipath. Each Node also has a Mellanox Infiniband Adapter to connect the Nodes to the rest of the VSC-3.
===== Measurement protocols =====
==== Peak performance ====
The peak performance was achieved with [[tools:benchmarks:ior:start|IOR]] running on the storage nodes (12 Processes per Node) and every process was forced to write on the local disks. So no interconnect was used (except for syncing the ior processes of course).
==== Sustained metadata performance ====
The metadata performance was measured with mdtest, running 128 processes distributed over 8 nodes.
mdtest-1.9.3 was launched with 128 total task(s) on 8 node(s)
Command line used: mdtest ./mdtest -I 5 -z 5 -b 5 -u -d /binfs/xxxx/xxxx/mdtest/
Path: /binfs/xxxx/xxxx/mdtest
FS: 105.9 TiB Used FS: 60.2% Inodes: 0.0 Mi Used Inodes: -nan%
128 tasks, 2499840 files/directories
SUMMARY: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
Directory creation: 7611.328 7611.328 7611.328 0.000
Directory stat : 203195.314 203195.314 203195.314 0.000
Directory removal : 4554.320 4554.320 4554.320 0.000
File creation : 47464.587 47464.587 47464.587 0.000
File stat : 206807.699 206807.699 206807.699 0.000
File read : 86282.435 86282.435 86282.435 0.000
File removal : 47321.744 47321.744 47321.744 0.000
Tree creation : 87.921 87.921 87.921 0.000
Tree removal : 43.238 43.238 43.238 0.000
==== Sustained performance ====
Performance of the system has been measured with [[tools:benchmarks:ior:start|IOR]] running on the 16 storage nodes with 12 processes / node.
The used BeeGFS configuration was:
* BeeGFS 6.7
* Striping 4 OSTs per File
* Chunksize 8MiB