User Tools

Site Tools


FS: Lustre

Characteristics

Description

The deployed file system is the Fujitsu Exabyte File System (FEFS), that is based on Lustre. Effectively, the system has two file systems, a file system local to the K-Computer with 11 PiB and a global file system with 30 PB [3]. The local file system consists of 2592 OSS (5184 OSTs) and the global file system of 90 OSS (2880 OSTs); performance of the local file system is 3.2 TB/s and 1.4 TB/s for read and write, respectively [3]. This is much higher than the global file system which provides a throughput of 0.2 TB/s. A staging mechanism transfers data before/after a job is completed from the fast local storage to the global file system. The local file system can only be accessed from compute nodes and not from login or post-processing nodes.

Measurement protocols

Peak performance

Sustained metadata performance

The reported metadata rate has been measured using mdtest on 9000 nodes with 100 files/node. See [3, slide 34]. Other benchmark runs have been conducted, for example, a mdtest on a variable number of clients shows how performance degraded with increase in clients and depends on the type of operation [2, slide 23].

Sustained performance

This was measured with the IOR benchmark using POSIX-I/O (90 OSS, 2880 OSTs). See [2] for a description.