no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
| — | hpsl:2020:deu:dkrz:lustre02 [2020/08/13 17:09] (current) – created - external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | ====== FS: Lustre02 ====== | ||
| + | ===== Characteristics ===== | ||
| + | |||
| + | < | ||
| + | name:Lustre Phase2 | ||
| + | </ | ||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | |||
| + | ===== Description ===== | ||
| + | |||
| + | The DKRZ system was procured in two phases that are roughly the same size. | ||
| + | The second phase consists of [[http:// | ||
| + | |||
| + | Both systems are configured in Scalable System Units (SSUs); pairs of servers in active/ | ||
| + | |||
| + | Initially, we planned of creating one big shared file system, but now are using two file systems (one for the storage of phase 1 and one for phase 2). Both file systems are mounted on all compute nodes. | ||
| + | |||
| + | ===== Measurement protocols ===== | ||
| + | |||
| + | ==== Peak performance ==== | ||
| + | |||
| + | The peak performance is derived from the maximum performance possible on a L300 that is 5.4 GiB/s, multiplied with the number of servers in the SSU/ | ||
| + | The L300 actually manages to achieve a better performance and operates at Infiniband speed. Still for the theoretic maximum, we consider the limit of 5.4 GiB/s. | ||
| + | |||
| + | Lustre' | ||
| + | |||
| + | ==== Sustained metadata performance ==== | ||
| + | |||
| + | Performance has been measured using [[tools: | ||
| + | The benchmark runs for a considerable time on 16 nodes with 16 processes per node but does not explicitly synchronize between the individual parabench runs. | ||
| + | Theoretically, | ||
| + | |||
| + | In phase 2, we received additional 7 metadata servers, they now delivered between 30 and 35k Ops/s if stressed individually resulting in 210 kOPS/s. | ||
| + | |||
| + | While both benchmarks have been executed individually, | ||
| + | |||
| + | ==== Sustained performance ==== | ||
| + | |||
| + | The reported performance result is only for the new phase 2 system. | ||
| + | |||
| + | Performance of the phase 1 system has been measured with [[tools: | ||
| + | |||
| + | Similarly performance of the phase 2 system has been measured. | ||
| + | The configuration was as follows: | ||
| + | * Striping 128 OSTs = 32 SSUs | ||
| + | * 852 compute nodes, 4 IOR procs per node | ||
| + | * Arguments to IOR: -b 2000000 -t 2000000 | ||
| + | * The amount of data was about 3x main memory of the used nodes | ||
| + | |||
| + | The measurement has been conducted while production in Phase 1 was active. Since both systems share the Infiniband tree network, the observed performance is lower than the system capabilities. | ||
