This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
io500:rules:submission [2020/05/22 20:34] – ISC 20 rules publishing from approved Google doc gflofst_sandia.gov | io500:rules:submission [2020/08/13 17:09] – external edit 127.0.0.1 | ||
---|---|---|---|
Line 3: | Line 3: | ||
The following rules should ensure a fair comparison of the IO-500 results between systems and configurations. They serve to reduce mistakes and improve accuracy. | The following rules should ensure a fair comparison of the IO-500 results between systems and configurations. They serve to reduce mistakes and improve accuracy. | ||
- | | + | For ISC20, submission of test runs should use new C-Application io500 which automatically runs both the new C version and the existing bash version (following the rules for SC-19) to ensure the consistency of results between the two implementations. An exception to this rule is possible for submitters who have a legitimate reason by requesting an exception from the committee via comittee@io500.org. |
Details are provided in the https:// | Details are provided in the https:// | ||
Line 9: | Line 9: | ||
- Submissions are made using the latest version of the IO500 application in GitHub which runs both the existing bash version and the new C version and all binaries should be built according to the included build instructions. | - Submissions are made using the latest version of the IO500 application in GitHub which runs both the existing bash version and the new C version and all binaries should be built according to the included build instructions. | ||
- $ git clone https:// | - $ git clone https:// | ||
- | | + | |
- Read-after-write semantics: The system must be able to correctly read freshly written data from a different client node after the close operation on the writer has been completed. | - Read-after-write semantics: The system must be able to correctly read freshly written data from a different client node after the close operation on the writer has been completed. | ||
- All create/ | - All create/ | ||
Line 19: | Line 19: | ||
- Submitting the results must be done in accordance with the instructions on our submission page. Please verify the correctness of your submission before you submit it. | - Submitting the results must be done in accordance with the instructions on our submission page. Please verify the correctness of your submission before you submit it. | ||
- If a tool other than the included pfind is used for the find phase, then it must follow the same input and output behavior as the included pfind and the source code must be included in the submission. | - If a tool other than the included pfind is used for the find phase, then it must follow the same input and output behavior as the included pfind and the source code must be included in the submission. | ||
+ | - It is not required to capture the list of matched files. | ||
- Please also refer to the README documents in the GitHub repo. | - Please also refer to the README documents in the GitHub repo. | ||
- Please read the CHANGELOG.md file for the new changes on the IO-500 benchmark | - Please read the CHANGELOG.md file for the new changes on the IO-500 benchmark | ||
Line 26: | Line 27: | ||
- For the 10 node challenge, there must be exactly 10 physical nodes and at least one benchmark process must run on each | - For the 10 node challenge, there must be exactly 10 physical nodes and at least one benchmark process must run on each | ||
- The only exception to this rule is the find benchmark which may optionally use fewer nodes/ | - The only exception to this rule is the find benchmark which may optionally use fewer nodes/ | ||
+ | - Each of the four main phases (IOR easy and hard, and mdtest easy and hard) has a subdirectory which can be precreated and tuned (e.g. using tools such as lfs_setstripe or beegfs_ctl); | ||
Please send any requests for changes to these rules or clarifying questions to our mailing list. | Please send any requests for changes to these rules or clarifying questions to our mailing list. | ||
Line 50: | Line 52: | ||
- Submitting the results must be done in accordance with the instructions on our [[https:// | - Submitting the results must be done in accordance with the instructions on our [[https:// | ||
- If a tool other than the included pfind is used for the find phase, then it must follow the same input and output behavior as the included pfind. | - If a tool other than the included pfind is used for the find phase, then it must follow the same input and output behavior as the included pfind. | ||
+ | - It is not required to capture the list of matched files. | ||
- Please also refer to the [[https:// | - Please also refer to the [[https:// | ||
- Please read the CHANGELOG.md file for the new changes on the IO-500 benchmark | - Please read the CHANGELOG.md file for the new changes on the IO-500 benchmark |