Sidebar
Table of Contents
FS: BINFL
Characteristics
Description
The BINFL (short for Bioinformatics large) storage system was procured in late 2016 consisting of 192 x Seagate ST8000NM0075 8 TB disks. It shares it's metadata SSDs with the BINFS-Filesystem. Running 8 metadata servers (buddy-mirrored) and 16 OSTs. The nodes are used as storage and compute nodes, therefore they are connected via Intel Omnipath. Each Node also has a Mellanox Infiniband Adapter to connect the Nodes to the rest of the VSC-3.
Measurement protocols
Peak performance
The peak performance was achieved with IOR running on the storage nodes (12 Processes per Node) and every process was forced to write on the local disks. So no interconnect was used (except for syncing the ior processes of course).
Sustained metadata performance
The metadata performance was measured with mdtest, running 128 processes distributed over 8 nodes.
mdtest-1.9.3 was launched with 128 total task(s) on 8 node(s) Command line used: mdtest ./mdtest -I 5 -z 5 -b 5 -u -d /binfl/xxxx/xxxx/mdtest/ Path: /binfl/xxxx/xxxx/mdtest FS: 1164.3 TiB Used FS: 0.0% Inodes: 0.0 Mi Used Inodes: -nan% 128 tasks, 2499840 files/directories SUMMARY: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- Directory creation: 7952.539 7952.539 7952.539 0.000 Directory stat : 209917.737 209917.737 209917.737 0.000 Directory removal : 4468.336 4468.336 4468.336 0.000 File creation : 44721.663 44721.663 44721.663 0.000 File stat : 208951.763 208951.763 208951.763 0.000 File read : 88973.752 88973.752 88973.752 0.000 File removal : 50546.614 50546.614 50546.614 0.000 Tree creation : 81.198 81.198 81.198 0.000 Tree removal : 48.698 48.698 48.698 0.000
Sustained performance
Performance of the system has been measured with IOR running on the 16 storage nodes with 12 processes / node.
The used BeeGFS configuration was:
* Striping 4 OSTs per File
* Chunksize 1MiB
