[BANANA] Berkeley Lab - Scientific Computing Seminar, September 21, 2006

[Parry Husbands]

Date: Thursday, September 21, 2006
Time: 1:00pm-2:00pm
Location: Building 50F-1647

Seminar Speaker: Yijian Wang
                         Department of Electrical and Computer Engineering
                         Northeastern University
                         http://www.ece.neu.edu/students/yiwang/

Title: Accelerating I/O-Bound Parallel Scientific Applications

Abstract:

In the area of high performance parallel computation, there is a growing
need to process very large, complex datasets. When executing these  
workloads
on cluster-based systems, performance cannot scale by simply increasing the
number of compute nodes. To effectively exploit parallel resources, we  
need to
parallelize file I/O. The potential impact of exploiting parallel I/O  
grows as the gap
between CPU and disk speeds continues to increase. While parallel I/O  
middleware
systems (e.g., MPI I/O) provide users with environments where large  
datasets can
be shared among multiple distributed processes, the performance of  
I/O-bound
applications depends heavily on how the data is accessed and where the  
data is
physically located on disk. I/O operations need to be parallelized both at  
the
application level and at the disk level.

In this talk, we present an I/O partitioning scheme that utilizes both  
local and centralized
I/O storage within a cluster to improve I/O parallelism and scalability.  
We characterize
file access patterns by studying I/O profiles. We propose and evaluate a  
new file-partitioning
algorithm to optimize data layout in cluster I/O subsystems. We also  
present a workload
balancing mechanism that targets on heterogeneous clusters and tunes  
workload assignments
to adapt the underlying storage devices. By migrating I/O workloads from  
heavily loaded nodes
to lightly loaded nodes, we are able to balance I/O workloads and achieve  
higher parallel I/O
throughput. Experimental results show that by utilizing our optimization  
approaches, I/O
performance can be improved significantly.

Sponsor of Seminar: David Skinner