Shared Parallel File System
Fangbin Liu
System and Network Engineering University of Amsterdam
Shared Parallel File System
• Introduction of the project
• The PVFS2 parallel file system
• The GFS cluster file system
• Comparison of PVFS2 and GFS
Shared Parallel File System
• Introduction of the project
• The PVFS2 parallel file system
• The GFS cluster file system
• Comparison of PVFS2 and GFS
Introduction of the project
• Parallel file system
– Industrial standard commodity PCs – NFS (Network File System)
– Many new shared file systems available: PVFS2, GFS, …
• The goal of the project
– Test PVFS and GFS file system on the test
Shared Parallel File System
• Introduction of the project
• The PVFS2 parallel file system
• The GFS cluster file system
• Comparison of PVFS2 and GFS
The PVFS2 parallel file system
• What is PVFS2 file system
– Parallel Virtual File System 2
– A remote storage server management system
• Internal organization
– Servers and clients
– Data storage organization – Two types of interfaces
The PVFS2 parallel file system
• Features supplied by PVFS2 file system
– Few dependencies
– Multiple interfaces supported – Modular design
– Distributed data storage
• Deployment of PVFS2 file system
– Generate the configuration file – Start up the servers
Shared Parallel File System
• Introduction of the project
• The PVFS2 parallel file system
• The GFS cluster file system
• Comparison of PVFS2 and GFS
The GFS cluster file system
• Introduction of GFS file system
– Managed by Red Hat Cluster Suite
– Supplies a uniform view among cluster nodes – Load distribution and balancing
– Installed directly on SAN or by GNBD devices – Built up with Logical Volumes created by
The GFS cluster file system
•
Deployment of GFS file system
1. Forming a cluster domain through cluster configuration tools
2. Forming a fencing domain through fence daemon
3. Export/Import GNBD device 4. Starting Cluster LVM daemon 5. Mount the GFS file system
Shared Parallel File System
• Introduction of the project
• The PVFS2 parallel file system
• The GFS cluster file system
• Comparison of PVFS2 and GFS
Comparison of PVFS2 and GFS
• Work theory
– Servers and Clients management – Storage management
– System consistency supports – Robust and scalability
Comparison of PVFS2 and GFS
• Test of performance
– Test conditions:
• 4 test nodes:
– CPU: Intel® Xeon™ CPU 3.40 GHz – Cache: 1024 KB
– Memory: 2075884 KB
• Organizations:
– 2 servers - 2 clients – 3 servers – 1 clients
Comparison of PVFS2 and GFS
• Test of performance
– Test sets: Write, ReWrite, Read, ReRead – Throughput test: multiple processes
concurrent operations
– Test programme: IOZONE
• With options:
– -A Automatic, all the file size (64 KB, …) and block size (4KB, …)
– -t # Throughput test with # process for alternative operations (write, read, …, random write, …)
Comparison of PVFS2 and GFS
Di f f er ence of Wr i t e Per f or mance f or Recor d s i ze 128 KB 0 200 400 600 800 1000 1200 64 1024 16384 262144 4194304 千 KB Fi l e M B / s e c l ocal 3 s er ver s
Comparison of PVFS2 and GFS
Di f f er ence of Read Per f or mance f or Recor d s i ze 128 KB 0 500 1000 1500 2000 2500 3000 64 1024 16384 262144 4194304 千 KB Fi l e M B / s e c l ocal 3 s er ver s
Comparison of PVFS2 and GFS
Di f f er ence of Wr i t e Per f or mance f or Recor d s i ze 128 KB 0 100 200 300 400 500 600 700 800 900 64 1024 16384 262144 4194304 千 KB Fi l e M B / s e c 3 s er ver s 2 s er ver s
Comparison of PVFS2 and GFS
Di f f er ence of Read Per f or mance f or Recor d s i ze 128 KB 0 500 1000 1500 2000 2500 3000 64 1024 16384 262144 4194304 千 KB Fi l e M B / s e c 3 s er ver s 2 s er ver s
Comparison of PVFS2 and GFS
64 25 6 1 024 4 09 6 16 384 6 553 6 26 21 44 4 16 64 256 102 4 4096 16 38 4 0 100000 200000 300000 400000 500000 600000 700000 800000 900000 1000000 kB/s ec kB file kB r ecor di ozone gf s node 15 wi t h 2 cl i ent 2 s er ver Wri t e per f or mance
900000-1000000 800000-900000 700000-800000 600000-700000 500000-600000 400000-500000 300000-400000 200000-300000 100000-200000 0-100000
Comparison of PVFS2 and GFS
6 4 2 5 6 1 0 2 4 4 0 9 6 1 6 3 8 4 6 5 5 3 6 2 6 2 1 4 4 4 64 1024 16384 0 100000 200000 300000 400000 500000 600000 700000 800000 900000 1000000 kB/s ec kB f i l e kB r ecor di ozone pvf s 2 node 15 wi t h 2 cl i ent 2 s er ver Wri t e per f or mance
900000-1000000 800000-900000 700000-800000 600000-700000 500000-600000 400000-500000 300000-400000 200000-300000 100000-200000 0-100000
Comparison of PVFS2 and GFS
64 25 6 10 24 40 96 16 38 4 65 53 6 26 21 44 10 485 76 41 94 30 4 4 64 1024 16384 0 100 200 300 400 500 600 700 800 MB/sec 千 KB File KB RecordGFS node 12 1 client 3 servers Write performance
700-800 600-700 500-600 400-500 300-400 200-300 100-200 0-100
Comparison of PVFS2 and GFS
6 4 2 5 6 1 0 2 4 4 0 9 6 1 6 3 8 4 6 5 5 3 6 2 6 2 1 4 4 1 0 4 8 5 7 6 4 1 9 4 3 0 4 4 64 1024 16384 0 500 1000 1500 2000 2500 3000 MB/sec 千 KB File KB RecordGFS node 12 1 client 3 servers Read performance
2500-3000 2000-2500 1500-2000 1000-1500 500-1000 0-500
Shared Parallel File System
• Introduction of the project
• The PVFS2 parallel file system
• The GFS cluster file system
• Comparison of PVFS2 and GFS
Project conclusion and future outlook
• The PVFS2 filesystem
– A popular parallel file system
– Stable performance – Multiple possibilities – User level stripping – Ease installation
• The GFS file system
– A reliable package
– Robust and scalability – Multiple configuration
forms
• Future outlook
