1 Introduction

Parallel tasks represented by Directed Acyclic Graphs (DAGs) referenced as Parallel Task Graphs (PTGs) and constituted by a set of subtasks of scientific and enterprise applications, require large amounts of processor and storage space during their execution in High Performance Computing Systems (HPC Systems) [¹], such as clusters [¹⁰], in order to achieve desired level of service [¹⁶] to users. Clusters or cluster computing refers to the use of interconnected computers geographically distributed [¹⁹, ⁸] as a high-performance computing platform and has become wide-spread [¹⁰, ³].

Often the computers used in a cluster are “commodity” computers, that is, low-cost personal computers as used in the home and office [³]; they consist of PCs heterogeneous, running Linux, and connected with Ethernet, and referred to as Beowulf clusters; since the processors are independent they are examples of the MIMD (Multiple Instruction, Multiple Data) or SPMD (Single Program, Multiple Data) model [⁷]. In this work, the processor refers to an individually programmable computer resource [⁵] that can be assigned to a PTG and in the rest of the paper are referred to as nodes, processing elements (PE) and resources. Resource is a collection of components that can be scheduled to perform an operation by an application [¹⁵]; some traditional examples of resources are CPU cores for computing, memory spaces for storage, network links for transferring, electrical power, and so on [¹⁸].

When Parallel Tasks Graphs in a cluster are executed, a scheduler performs the scheduling and the allocation resources [¹⁰]. The scheduling is performed in two phases: first, a search for idle resources in all clusters is performed to determine the positions of the PEs remain idle; the search for idle resources can be performed by a resource allocation control which is a mechanism that manages and control resources in a cluster system [¹⁰]. Second a search for the best allocations to optimize the execution times of the tasks is performed.

In [²¹], the open access article, an array-based scheduler (called the Array Method) for scheduling and resource allocation in clusters is explained. For scheduling, the Array Method uses an array of idle resources or processors (resource array) that is updated as follows, each time a cluster is added to the target system, a task finishes executing or a PE of a cluster is added as a system resource; for locating idle resources, a sequential search is performed on each of the clusters of the HPC System.

The search is a scan that is performed for each of the processors belonging to the clusters, generally initiated on a central node and executed by a Software Agent (SA); the SA initiates the scan on all the processors belonging to the cluster; upon finishing with a cluster, the SA advances to the neighboring clusters and starts again a similar scanning process, processor by processor; every so often SA informs the central node the number of visited processors and activates a process to update the idle resource matrix (resource matrix). The SA resides in the central node of the HPC Systems and because resources can change from a busy state to an idle state at any time, SA should start the sequential search for idle resources in the clusters and update the resource matrix periodically and continuously. Both processes become fundamental and necessary. Finally, for the assignments of idle resources to tasks, a sequential search is executed in the resource matrix and the process of searching is initiated for the best assignments.

Considering the results of the execution times generated by the matrix method and an analysis of each process executed by the scheduler, the most time-consuming process is the sequential scanning of idle resources in the clusters. In order to improve the resource search times in the clusters that constitute the target system, this paper present a technique of idle resources search with Lévy flights, a special class of random walks that have been investigated in different works [¹⁹, ¹⁵, ²², ⁴, ¹³, ²³, ¹¹] as a resource search technique in locations geographically distributed; in the following paragraphs the development and implementation of this technique is justified. The term Lévy random walks is used to refer Lévy flights by context in which this technique is used in this research work.

2 Basic Definitions

In this paper, the section of basic definitions of [²¹] is used and the following new definitions are proposed.

Definition 1. The target system consists of Cl clusters, C1,C2,…,Cl where l is the number of clusters contained in the HPC System. Each cluster contains m heterogeneous processors with n processing cores, then Cl,m,n is cluster k, processor m, processing core n. In this way each Cl,m,n represents a resource that is identified as Rl,m,n and can be used by the scheduler.

Definition 2. Let to consider definition 2 from [²¹]; it follows that each PTG will request ηi resources, which must be extracted from the scheduler’s resource array, then the next condition is established:

For the PTG to complete its execution.

Definition 3. A Lévy flight is a type of random walk in which the increments are distributed according to a ”large tail” probability distribution. Specifically, the distribution used can be approximated by a power law of the form [²³]:

where μ is a constant parameter of the distribution known as the exponent or scaling parameter.

Definition 4. Let a software agent SA, which locates any idle Rl,m,n found in any Cl using equation 2, then registers it in an array of resources and condition 1 is satisfied.

3 Problem Statment

Let a scheduler running on an HPC System consisting of Cl clusters, C1,C2,…,Cl where l is the number of clusters of the HPC System. Each cluster contains m heterogeneous processors with n processing cores; for searching R1,R2,…,Rn idle resources, a search engine S operated by a software agent (SA), is executed using equation 2.

Then: respecting the PTGs execution constraints, β must be executed continuously and periodically at times t1,t2,…,tn, where tn, is the time where the scheduler is active in the target system. Any R1,R2,…,Rn idle resource located by S must be stored in an array of dimension N×M and be available for use at any time tn.

4 Related Works

In a plethora of literature, research works that use idle resources for scheduling and allocation of tasks in an HPCS, assume a set of resources that are available in a resource pool as in [¹⁹, ¹⁵]; based on required characteristics, the resource discovery mechanism searches and returns the addresses of the resources that match with the provided descriptions [¹⁵], in [¹] resources are scheduled prior to path setup request, the allocator has a resources’ metadata [²]; in [²⁰] assume that each user contributes a certain number of machines (resources) to its common pool of machines (resources) in the cloud, [⁸] only proposes a resource management system manage a pool of processing elements (computers or processors) which is dynamically configured (i.e., processing elements may join or leave the pool at any time); other research works such as [¹⁴] establishes a model where capabilities of the machines are known: the number of cores, the amount of memory, the disk space, and the host OS running, and supposes that a large data center and cloud systems already have significant monitoring tools that provide near-real-time updates of various systems to their controllers. In [¹⁸] the resources are considered to be available on demand, charged on a pay-as-you-go basis, and in one aspect, cloud providers hold enormous computing resources in their data centers, while in the other aspect, cloud users lease the resources from cloud providers to run their applications; [⁹] the cloud service end user can use the entire stack of computing services, which ranges from hardware to applications. [¹⁷] supposses a cloud of resources where tasks are scheduled, the resources are heterogeneous and characterized by power and cost constraints. With the resources, metrics such as makespan, throughput [¹⁵], waiting time [¹⁹, ¹⁶, ¹⁰], overall mean task response time [⁸], load balancing [¹²] are sought to improve in HPC Systems.

In contradistinction to the works described above, in this research work any set of resources for the schedule are not assumed, but proposes the development of idle resources search engine operated by a software agent; software agent executes a resource discovery procedure [¹⁹, ¹⁵], which allows visiting a set of available clusters in the shortest possible time.

Considering Lévy random walks [²², ⁴, ⁶] as a prominent area of research in various disciplines from ecology to physics [²³], and a special class of random walks whose stride lengths are not constant but are selected from a probability distribution with a power law [²², ⁴, ¹³, ²³], in addition to the hypothesis that Lévy random walks are optimal when exploring unpredictably distributed resources [¹³] have been proposed in this work as a strategy of search for idle resources in an HPC System. Similarly, Lévy’s random walks are used in this work as a search strategy, because due to the divergence of the variance, extremely long jumps can occur and the typical trajectories are self-similar in all scales, showing groups of jumps shorter interspersed [⁶].

5 Array Method

The array method [²¹], is a dynamic scheduler for scheduling and allocating tasks in an HPC system. The operation of this scheduler is based on a set of arrays that store the results generated from each function. For each of the arrays, iterative processes or loops are executed. A very brief review of the data structures used by the Array Method is presented below.

The software agent SA, is executed over all clusters; all characteristics of idle resources found are stored in resource matrix, i.e., when the SA is executed, it only updates the resource matrix so the planning and allocation process can use idle resources.

The Software Agent execution has been defined so far. On the next paragraphs Lévy random walks and its SA interaction with, will be defined; how this interaction operates over the clusters are defined too.

6 Lévy Random Walks

This section defines how Lévy random walks are used in this research work. The exponential increase in step length gives the Lévy flight the property of being scale invariant, and they are used to model data exhibiting clustering. Lévy flights are a special class of random walks whose step lengths are not constant, but are selected from a probability distribution with a power law [²², ⁴, ¹³, ²³]. The random walk is represented by a succession of random variables Xn with n∈N known as a discrete stochastic process. The sequence Xn,n∈N forms infinite sequences X0,X1,…,Xi with Xi∈Z. That is, if a run starts at state 0, at the next time the agent can move to position +1, with probability p or to position −1 with probability q, with p+q=1. Position +1 is considered the PE to the left of the current PE and position −1 is considered the PE to the right of the current PE.

In this research work we analyze the case of a software agent which starts from a specific cluster and PE, and moves in stages or steps along the target system, at each step it moves a unit distance to the right or to the left, with respectively equal probabilities. For the movement of the agent, let ζn as the nth motion of the agent and P as the probability function then,

ζn are considered as the independent random variables.

It is further assumed that the agent moves k steps in total, before registering idle resources in the resource matrix (see section 5). For these experiments, Lévy flights are only used as a search procedure for idle resources, not as a prediction algorithm. The steps for the execution of the software agent are explained on next section (a reduced algorithm is provided in Table 1).

6.1 Search for Idle Resources in HPC System using Lévy Random Walks

For the resource allocation in HPC Systems, most of research works (as Related Works section explains) assume a centralized resource manager that has a complete vision of network topology, as well as networking and computing resources status; this assumption is not valid for large-scale worldwide grid networks; practically, grid network comprises geographically distributed heterogeneous resources interconnected by multi-domains networks [¹]. In this paper, before performing the scheduling and allocation resources, a resource search engine is proposed in HPC Systems, using Lévy random walks to extract the characteristics of each processing element, assuming that ignoring computational resources capacity and availability may affect the overall performance significantly specially in computational intensive applications [¹]. To generalize the problem, lets consider the figure 1, which has a set of geographically dispersed clusters linked by wireless and wired communication; figure 1 is the most similar to actual architecture used in the experimentations. For reasons of space, details about network band-width, multidomain environments, interdomain, and intradomain topology are not specified, in addition to how the different domains interact to provide end-to-end connectivity.

Fig. 1 HPC Systems with clusters showing a search for idle resources using Lévy random walks 

On figure 1 circles represent processing elements; filled circles show occupied PEs, circles filled with lines represent PEs without functionality, while unfilled PEs are idle at time t+1 and can be located by the software agent. Links between processing elements within a cluster and links between clusters are represented by solid lines. The three dots above the solid lines show the possible addition of new clusters to the target system.

The example showed in the Figure 1 up operates like this: the resources search for in the HPC System using Lévy random walks, is shown with a dotted line; the Lévy random walk executed by software agent using algorithm showed in Table 1, starts in cluster 1, executes a set of steps for searching the PEs on this cluster, migrates to cluster 2, 3, 4 and 5; on the cluster 5 a long jumps is executed. The search process is repeated for the time set in the algorithm; at the end of the random walk, twenty idle resources are identified by the SA, resource matrix is updated and resources are available for scheduling and allocation.

Considering the algorithm proposed in [¹¹], Table 1 shows a reduced structure of the resource search algorithm using Lévy’s random walks, operated by the software agent. On next sections the results of the experiments performed are described.

7 Experiments

Target System. Experiments are performed on 450 desktop computers and a server farm with 10 servers. Clusters between 5, 10 and 20 computers were built and distributed in different buildings within the campus.

Algorithm 1 Algorithm for resources search 

The server farm is considered the initial cluster and where the task queue remains.

Agent Software. The software agent is programmed with C language, and its movements are allowed through C shell programming within the target system. Once the agent is transported to a PE, the execution of the agent is performed with a Linux Cron utility programmed in each EP to detect the arrival or presence of the agent in the PE.

Considering the purpose of this survey is the performance analysis of the resources search techniques, two metrics for the experiments in this paper are used:

Fig. 2 The percentage of resources located by search engine operated by SA in the target system 

Each experiment and the results obtained are explained in the following paragraphs.

The percentage of resources located by each algorithm in the target system at time t. In these experiments two types of searches are executed as follows: from the server farm and from where the last search ended; with these experiments it is possible to measure the ability of each technique to locate the resources in the target system, in time t.

For each of these experiments, numbered 1 to 20, a set of 100 tests is run; in 50 tests the starting position of the search is in the initial cluster and 50 tests the starting position is the location of the last search; alternating the experiments, a test is run with the first type of search and then with the second type of search.

For experiment 1, the tests are started with a time t of 5 minutes, 100 experiments are run, the result of each of the 100 runs is the number of resources found at time t (5 minutes); the total sum of the resources found in each run is divided by 100, and the result is reported for this experiment. For a second experiment the time t is 10 minutes and the 100 runs, the results are obtained in the same way as experiment 1. For each experiment 5 minutes are incremented. The results of all experiments are shown in graph 2.

Results (Experiment 1). The results obtained in these experiments show that the random walk search is more efficient for locating geographically distributed resources, selected with a probability distribution; this search method allows moving to clusters that may be without assigned tasks (completely idle).

During the execution of this search method, subgroups with different amounts of free resources were found in the clusters, before the migration of the SA from one cluster to another; with these groups, the system can send tasks with a number of subtasks that are equal to the subgroups of resources found, allowing the execution of the tasks with their subtasks in the same cluster.

In contrast to first experiment, we propose an experimentation related to the way in which the tasks are disaggregated into the resources found by each type of search executed: the sequential search and the search using Lévy flights. The following paragraphs explain this experimentation.

Disaggregation of the tasks in the HPC System clusters, during the resource allocation process. Once the resources are located the resource matrix and the matrix of characteristics are updated; the next step of the algorithm is to assign the tasks to the available resources.

For this experimentation, the total execution time of the tasks was measured considering that Lévy random walks searches can locate resources with longer geographic distances. When the subtasks of a task are executed in geographically distant clusters with long distances, the communication times between tasks increase gradually, depending on the speed of the network devices of the target system. The workloads of [²¹] were chosen for this experiment.

The tasks of each workload were scheduled and allocated to the resources that each algorithm found in the HPC System. The completion times of the tasks of each workload were measured considering the communication time between subtasks of each task. The results of these experiments are shown in graph 3.

Results (Experiment 2). Results obtained in the experiments of scheduling and allocation of localized resources by each method have shown the outperformance of the Lévy random walks, since lower task execution times are obtained.

More resources that this technique can locate in the resource search executions, allow to make more task assignments in geographically distributed clusters; assigning tasks with their subtasks grouped in the same cluster can decrease communication times and the task execution is faster. When scheduling is executed, the scheduler can have more available resources for assignment, and allocation can be done in close clusters of resources.

Fig. 3 The percentage of resources located by each search engine in the target system at time t 

8 Conclusion and Future Work

The emulation in computational environments of scale-invariant phenomena observed in biological systems has led to propose solutions to problems of the non-deterministic polynomial time type. In this research work we conducted an exploratory study to propose a solution that allows the search for idle resources in an HPC System, specifically in a cluster system using Lévy random walks.

The justification of this work is the development of a software agent which migrates between clusters of an HPC system and simulates Lévy random walks. With this software agent a comparison is made with two performance metrics between a sequential search of resources and Lévy random walks: the number of resources located in a given time, and in contrast the total execution time of the tasks using the resources located by each technique is measured.

In the experiments with the first metric, Lévy random walks allow visiting clusters in geographic distances that a sequential resource-by-resource search would take a longer amount of time. With the second metric, the total execution time of the tasks using the resources obtained with each algorithm of search has been evaluated; this total time is measured from the assignment of the resources to the task and the dispatch of the task, until the task finishes its execution and returns to the central node.

In the experiments, it is verified that, the total times generated with the resources obtained with the Lévy random walks’ algorithm, are not so far from the total times obtained with the resources located in the sequential search, when the workloads contain less than 500 PTGs for their processing; shorter total execution times of the tasks are highlighted when the workloads are dense.

Finally, it should be noted that the resources, architecture and configuration of the computer network can affect the performance of the algorithms when the configurations are not adequate.