Apache Mesos

From Bauman National Library
Revision as of 13:35, 15 December 2017 by egor zorin (Talk | contribs) (Architecture)

Apache Mesos
Developer(s) Apache Software Foundation
Stable release
1.4.0 / 18.09.2017
Repository {{#property:P1324}}
Development status Active
Written in C++
Operating system Cross-platform
Type Computer cluster management
Website mesos.apache.org

This is a centralized fault-tolerant cluster management system designed for distributed computing environments in order to provide resource isolation and easy management of clusters of subordinate nodes (mesos slaves). Apache Mesos is an open source system that provides efficient isolation of resources and the sharing of distributed applications or structures. [Source 1]
Apache Mesos is an open cluster manager that simplifies the launch of applications on a scalable server cluster and is the heart of the Mesosphere system.


Mesos was formed as a research project at the University of California Berkeley (UC Berkeley Lab) graduate students Benjamin Hindman, Andy Konvinsk and Mateja Zacharias, as well as Professor Ian Stoic. Students began to work together on the project within the framework of the course on Advanced Topics in Computer Systems, taught by David Kuller. It was originally named Nexus, but due to a conflict with another university project, it was renamed Mesos.

Mesos was first introduced in 2009 by Andy Konvinski on HotCloud '09. Later in 2011, it was presented in a more mature state in a report by Mateja Zacharias at the USENIX Symposium on Networking Systems for the Design and Implementation of a Conference on the Work "Mesos: A Platform for Distributed Resource Sharing in the Data Center" by Benjamin Hindman, Andy Konvinski, Mateja Zacharia, Ali Godsie, Anthony D. Joseph, Randy Katz, Scott Schenker, Ian Stoic.

How it works

In a sense, the essence of Apache Mesos' work is the opposite of traditional virtualization: instead of dividing a physical machine into a set of virtual machines, it is suggested to combine them into a single whole, into a single virtual resource.

Mesos distributes CPU and memory resources in the cluster for tasks in a similar manner, as the Linux kernel allocates iron resources between local processes.

If you need to perform different types of tasks, you can select separate virtual machines (a separate cluster) for each type. These virtual machines will probably not be fully loaded and will stand idle for a while, that is, they will not work with maximum efficiency. If all virtual machines for all tasks are combined into a single cluster, you can increase the efficiency of resource utilization and, at the same time, increase the speed of their execution (if short-term or virtual machine tasks are not fully loaded all the time). thumb|800px|center| Pic. 1 Efficiency of Mesos A cluster Mesos (with a framework to it) is able to re-create individual resources, in case of their fall, scale resources manually or automatically under certain conditions, etc.


Mesos has many functions:

  • Scalability (up to 10,000 nodes);
  • Isolation of resources through Linux Containers;
  • effective planning of CPU and memory resources;
  • Web interface for monitoring cluster status;
  • Apache ZooKeeper;
  • High availability of servers.


The architecture of Apache Mesos consists of the demons master and slave (that is, leading and slave demons) and the framework.

A brief overview of these components and some important terms:

  • The master daemon (or Master daemon) runs on the master node (or node) and manages the slave daemons.
  • The slave daemon (or Slave daemon) runs on the master node and performs the tasks of the framework.
  • The framework (or the Mesos application) consists of a scheduler that combines with the master node to receive the resource offers of executors running tasks on the slave nodes. Examples of Mesos frameworks are Marathon, Chronos and Hadoop.
  • Offer is a list of available CPU resources and memory slave nodes. All slave nodes send offers to the master node, which passes them to the available frameworks.
  • Task - the unit of work planned by the framework that is executed on the slave node. The task can be anything, starting with a command or bash script and ending with SQL queries and Hadoop processes.
  • Apache ZooKeeper (or ZK): the software that is used to coordinate the master node.

  With these components, Apache Mesos accurately distributes cluster resources between applications according to their requirements. Cite error: Closing </ref> missing for <ref> tag
Cite error: <ref> tags exist for a group named "Source", but no corresponding <references group="Source"/> tag was found, or a closing </ref> is missing