Loops are the largest source of parallelism in many scientific applications. Parallelization of irregular loop applications is a challenging problem to achieve scalable performance on cluster and cloud systems. For cluster systems, previous research proposed an effective Master-Worker model on clusters for distributed self-scheduling schemes that apply to parallel loops with independent iterations. However, this model has not been applied to large-scale clusters. Cloud computing infrastructure offers computing resources as a collection of virtual machines by different hardware configurations, which is transparent to end users. In fact, the computational rates of these virtual machines instances are different and the system behaves as a heterogeneous environment. Thus, scheduling and load balancing for high performance computations become challenging issues.
We propose a hierarchical distributed approach suitable for scheduling parallel loops . We implemented our scheme on a large scale homogeneous cluster and also on a cloud system heterogeneous environment using the message passing interface protocol. We evaluated various performance aspects associated with our distributed scheduling scheme. In the future we plan to further investigate how cloud system features affect the performance of our approach and also to pursue implementation using the map reduce approach.