File System Workload Analysis for Large Scale Scientific Computing Applications [electronic resource].
- Published
- Washington, D.C. : United States. Dept. of Energy, 2004.
Oak Ridge, Tenn. : Distributed by the Office of Scientific and Technical Information, U.S. Dept. of Energy. - Physical Description
- PDF-file: 16 pages; size: 0.1 Mbytes
- Additional Creators
- Lawrence Berkeley National Laboratory, United States. Department of Energy, and United States. Department of Energy. Office of Scientific and Technical Information
Access Online
- Restrictions on Access
- Free-to-read Unrestricted online access
- Summary
- Parallel scientific applications require high-performance I/O support from underlying file systems. A comprehensive understanding of the expected workload is therefore essential for the design of high-performance parallel file systems. We re-examine the workload characteristics in parallel computing environments in the light of recent technology advances and new applications. We analyze application traces from a cluster with hundreds of nodes. On average, each application has only one or two typical request sizes. Large requests from several hundred kilobytes to several megabytes are very common. Although in some applications, small requests account for more than 90% of all requests, almost all of the I/O data are transferred by large requests. All of these applications show bursty access patterns. More than 65% of write requests have inter-arrival times within one millisecond in most applications. By running the same benchmark on different file models, we also find that the write throughput of using an individual output file for each node exceeds that of using a shared file for all nodes by a factor of 5. This indicates that current file systems are not well optimized for file sharing.
- Report Numbers
- E 1.99:ucrl-conf-201895
ucrl-conf-201895 - Subject(s)
- Other Subject(s)
- Note
- Published through SciTech Connect.
01/13/2004.
"ucrl-conf-201895"
Presented at: NASA / IEEE MSST2004, Adelphi, MD, United States, Apr 13 - Apr 16, 2004.
Miller, E; Brandt, S A; Long, D; Wang, F; Hong, B; Xin, Q; McLarty, T. - Funding Information
- W-7405-ENG-48
View MARC record | catkey: 14344892