====== FS: BINFL ======

===== Characteristics =====
 
<data_cdcl>
name:BINFL
</data_cdcl>

===== Description =====

The BINFL (short for Bioinformatics large) storage system was procured in late 2016 consisting of 192 x Seagate ST8000NM0075 8 TB disks. It shares it's metadata SSDs with the [[binfs|BINFS]]-Filesystem. Running 8 metadata servers (buddy-mirrored) and 16 OSTs. The nodes are used as storage and compute nodes, therefore they are connected via Intel Omnipath. Each Node also has a Mellanox Infiniband Adapter to connect the Nodes to the rest of the VSC-3.

===== Measurement protocols =====

==== Peak performance ====

The peak performance was achieved with [[tools:benchmarks:ior:start|IOR]] running on the storage nodes (12 Processes per Node) and every process was forced to write on the local disks. So no interconnect was used (except for syncing the ior processes of course). 

==== Sustained metadata performance ====

The metadata performance was measured with mdtest, running 128 processes distributed over 8 nodes.
<code>
mdtest-1.9.3 was launched with 128 total task(s) on 8 node(s)
Command line used: mdtest ./mdtest -I 5 -z 5 -b 5 -u -d /binfl/xxxx/xxxx/mdtest/
Path: /binfl/xxxx/xxxx/mdtest
FS: 1164.3 TiB   Used FS: 0.0%   Inodes: 0.0 Mi   Used Inodes: -nan%

128 tasks, 2499840 files/directories

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   Directory creation:       7952.539       7952.539       7952.539          0.000
   Directory stat    :     209917.737     209917.737     209917.737          0.000
   Directory removal :       4468.336       4468.336       4468.336          0.000
   File creation     :      44721.663      44721.663      44721.663          0.000
   File stat         :     208951.763     208951.763     208951.763          0.000
   File read         :      88973.752      88973.752      88973.752          0.000
   File removal      :      50546.614      50546.614      50546.614          0.000
   Tree creation     :         81.198         81.198         81.198          0.000
   Tree removal      :         48.698         48.698         48.698          0.000
</code>
==== Sustained performance ====

Performance of the system has been measured with [[tools:benchmarks:ior:start|IOR]] running on the 16 storage nodes with 12 processes / node.

The used BeeGFS configuration was:

* Striping 4 OSTs per File

* Chunksize 1MiB