Overview

In the context of the CRC 901, software composed of several components is executed in OTF Compute Centers.  Such components may create considerable data traffic inside a compute center. One option is to provide data centers with large capacity, i.e., a full-bisection bandwidth network. While this is conceptually simple, it is often not economically desirable. Hence, typical data-center networks are under dimensioned.  This might lead to congestion which in turn  slows down the whole execution process. So we need, for example, routing algorithms and traffic engineering techniques to prevent congestion. 
To conduct research on data centers either a large physical testbed or appropriate models and simulation (or emulation) tools are required. As a physical testbed of appropriate size is far too expensive we developed MaxiNet. MaxiNet is a highly scalable, distributed emulation environment for large Software-Defined Networks (SDN). MaxiNet runs on a pool of multiple physical machines called workers. Each of these workers emulates a part of a network. Traffic between the different parts is tunneled through a physical network interconnecting the workers. With an increasing number of workers, even larger networks can be emulated. Using Maxinet, we were able to emulate a data center consisting of 3600 nodes using only 12 physical workers.

As a showcase for MaxiNet, we used it to extend the Yarn Scheduler Load Simulator (SLS) with a realistic network model.  SLS is a framework to test novel scheduling algorithms for the Map/Reduce implementation Hadoop. SLS uses job traces from real world Hadoop Clusters together with the description of new scheduling algorithms and applies these algorithms to the job trace. With our extension, it will be possible to test novel scheduling algorithms that incorporate network properties into the scheduling process of Hadoop (Hadoop being one real world example of a composed software running in a data center). 

Contact

The MaxiNet Tool has been developed by subproject A2. If you have any questions, please contact research staff from Subproject A2.

Further Information: 
http://www.cs.uni-paderborn.de/?id=maxinet 
http://hadoop.apache.org/docs/r2.3.0/hadoop-sls/SchedulerLoadSimulator.html