Nutanix relies on the following services to run…
An Acropolis follower runs on every CVM with an elected Acropolis leader. The Acropolis follower is responsible for statistic collection and publishing and provides VNC proxy capabilities. The Acropolis leader is responsible for stat collection and publishing, task scheduling and execution, VM placement and scheduling, network controller, and VMC proxy.
Genesis is a process that runs on each node and is responsible for any services interactions (start/stop/etc.) as well as for the initial configuration. Genesis is a process that runs independently of the cluster and does not require the cluster to be configured/running. The only requirement for Genesis to be running is that Zookeeper is up and running.
Zookeeper stores information about all cluster components (both hardware and software), including their IP addresses, capacities, and data replication rules, in the cluster configuration. Zookeeper has no dependencies, meaning that it can start without any other cluster components running.
Zookeeper is active on either three or five nodes, depending on the redundancy factor (number of data block copies) applied to the cluster. Zookeeper uses multiple nodes to prevent stale data from being returned to other components. An odd number provides a method for breaking ties if two nodes have different information. Of these nodes, Zookeeper elects one node as the leader. The leader receives all requests for information and confers with its follower nodes. If the leader stops responding, a new leader is elected automatically.
Zeus is an interface to access the information stored within Zookeeper and is the Nutanix library that all other components use to access the cluster configuration.
A key element of a distributed system is a method for all nodes to store and update the cluster’s configuration. This configuration includes details about the physical components in the cluster, such as hosts and disks, and logical components, like storage containers.
Distributed systems that store data for other systems (for example, a hypervisor that hosts virtual machines) must have a way to keep track of where that data is. In the case of a Nutanix cluster, it is also important to track where the replicas of that data are stored.
Medusa is a Nutanix abstraction layer that sits in front of the database that holds metadata. The database is distributed in a ring topology across multiple nodes in the cluster for resiliency, using a modified form of Apache Cassandra.
Nutanix’s implementation of Cassandra uses a version of Apache Cassandra that has been modified for high performance and automatic, on-demand scaling. Cassandra stores all metadata about the guest VM data in a Nutanix storage container.
Cassandra runs on all nodes of the cluster. Cassandra monitor Level-2 periodically sends a heartbeat to the daemon, which includes information about the load, schema, and health of all the nodes in the ring. Cassandra monitor L2 depends on Zeus/Zk for this information.
A distributed system that presents storage to other systems (such as a hypervisor) needs a unified component for receiving and processing data that it receives. The Nutanix cluster has a software component called Stargate that manages this responsibility.
All read and write requests are sent across an internal vSwitch to the Stargate process running on that node. Stargate depends on Medusa to gather metadata and Zeus to gather cluster configuration data. From the perspective of the hypervisor, Stargate is the main point of contact for the Nutanix cluster.
A Curator leader node periodically scans the metadata database and identifies cleanup and optimization tasks that Stargate should perform. Curator shares analyzed metadata across other Curator nodes. The Curator depends on Zeus to learn which nodes are available, and Medusa to gather metadata. Based on that analysis, it sends commands to Stargate.