Salt as a Cloud Controller - Naukri Code 360

Introduction

Have you worked on the deployment of complex IT Systems? Have you ever worked on the automation of the configuration of IT Systems?

This article focuses on one of the automation engines, i.e., Salt Project. We will study the Cloud Controller part of Salt in detail. We will try to understand hypervisors and Virtual Machines as well. We will also see how to migrate a VM.

Salt The Cloud Controller

The Salt Version 0.14.0 has an enhanced cloud control mechanism.

It allows the direct management of private cloud VMs using Salt. The common name for this system is Salt Virt.
The Salt Virt system is already present and set up within Salt. This means that you don't require any other salt code deployments than setting up Salt.
Remember that you need the certtool binaries and the libvirt Python module.
Salt Virt's major objective is to provide a swift, straightforward, fully functional cloud that can scale.
The features of Salt Virt also include strong image and disc management, virtual machine migration with or without shared storage, and the ability to set up and manage sophisticated virtual machine networking are all included with Salt Virt.
In other words, you can use the Salt Virt to build a cloud from a blade center or a SAN. But it can also build a cloud from a collection of Linux desktops without a single shared storage system.
Salt Virt can create clouds using technology that is genuinely a commodity while still supporting the power of specialist hardware.

Working with the Hypervisors

The first and most important step is setting up the hypervisors. You need to set it up by installing the right version of the software. After the installation, you have to work on the configuration of the interface.

Installation of the HyperVisor Software

Salt Virt is designed to be hypervisor independent, although, at the moment, KVM via libvirt is the only fully functional hypervisor.

Keep in mind that libvirt and kvm are necessary software components for a hypervisor. You can also install either qemu-nbd or libguestfs for more enhanced features.

You can mount the Virtual Machine images before starting and pre-seed them with configurations and a salt-minion using the Libguestfs and qemu-nbd.

The sls will execute the procedures to configure the libvirt pki keys and set up the required software for a hypervisor.

libvirt:
  pkg.installed: []
  file.managed:
    - name: /etc/sysconfig/libvirtd
    - contents: 'LIBVIRTD_ARGS="--listen"'
    - require:
      - pkg: libvirt
  virt.keys:
    - require:
      - pkg: libvirt
  service.running:
    - name: libvirtd
    - require:
      - pkg: libvirt
      - network: br0
      - libvirt: libvirt
    - watch:
      - file: libvirt

libvirt-python:
  pkg.installed: []

libguestfs:
  pkg.installed:
    - pkgs:
      - libguestfs
      - libguestfs-tools

Configuring the HyperVisor Network

A network bridge must be running on the hypervisors if you want virtual machines to access network devices. The following script will create a standard bridge on the hypervisor and connect it to eth0:

eth0:
  network.managed:
    - enabled: True
    - type: eth
    - bridge: br0

br0:
  network.managed:
    - enabled: True
    - type: bridge
    - proto: dhcp
    - require:
      - network: eth0

Configuring the Virtual Machine Network

One of the features of Salt Virt is a method for simulating the network interfaces used by deployed virtual machines. Only one interface is made by default for the deployed virtual machine, bridged to br0. In case you want to start using the default networking configuration, make sure that the bridge interface br0 is present on the hypervisor and bridged to an operational network device.

The Libvirt State

Distributing libvirt certificates is one of the difficulties in setting up a cloud that is based on that library. Virtual machine migration is possible with these certificates. Salt has an automated technique for deploying these certificates. When creating keys for libvirt clients on the master, Salt handles the signing authority key, signs them with the certificate authority, and distributes them via pillar. The libvirt state controls this functionality. To verify that the certificate is present and current, just use the following calculation on the minion:

libvirt_keys:
  virt.keys

Getting The Virtual Machines Ready

Since Salt can't create virtual machine images instantly, Salt Virt needs that they should be provided. The underlying platform affects how these virtual machine images are generated.

Although it is possible to manually produce virtual machine images by using the KVM and running through the installer, this method is not advised because it is a manual process and prone to mistakes.

Applications for virtual machine generation are available across a variety of platforms:

You can work with kiwi, and
Vm-builder without any difficulties.

Putting virtual machine images in the Salt file server is the simplest approach to make them accessible to Salt Virt once they are ready. Salt Virt may now use an image by simply copying it into /srv/salt.

Existing Virtual Machines Images

Virtual machine images for Salt Virt are available from many different Linux distributions. You can use the images as per your convenience.

DISCLAIMER: SALTSTACK DOES NOT SUPPORT ANY OF THESE IMAGES.

CENTOS: These images are developed by OpenNebula, but they should work fine with Salt Virt. You only need the qcow file.
FEDORA LINUX: These images should work fine with Salt Virt as well.
OPENSUSE: These images should work fine with Salt Virt as well.
SUSE: These images should work fine with Salt Virt as well.
UBUNTU LINUX: These images should work fine with Salt Virt as well.

Working with SALT VIRT

Salt can now begin issuing cloud instructions using the virt runner once you have configured the hypervisors and the virtual machine images are ready.

Run the Salt Virt hypervisor info command first:

salt-run virt.host_info

This will conduct a stats query on the active hypervisor(s) and display valuable data like the number of CPUs and RAM.

Additionally, you may make a list of every VM and its state across every hypervisor node:

salt-run virt.list

As soon as a hypervisor becomes accessible, you can create a new virtual machine using the virt.init routine:

salt-run virt.init centos1 2 512 salt://centos_test.img

Now, the Salt Virt runner will choose a hypervisor on its own to deploy the new virtual machine on. When using salt:/, the Salt File Server on the master is assumed that the CentOS virtual machine image is present in its root directory. The virt:images config parameter determines the destination directory of the images on the hypervisor minion when images are copied locally after being retrieved from the file server. By default, this is /srv/salt-images/.

When a virtual machine (VM) is started using "virt.init," a copy of the image is made in the hypervisor using the cp.cache file. Then it will be mounted and seeded with a minion. Salt will also install the Pre-authenticated keys on the new machine as part of the seeding process. When using the default option to "seed.apply," a minion will be installed if one cannot be located on the image.

When the installation of Salt Minion has become necessary, the delay in launching VM becomes highest. Virtual machine deployment will significantly speed up by ensuring that the source VM images already have Salt installed.

By explicitly using the virt execution module with an absolute path of the image, you can also deploy an image on a specific minion. This is pretty useful for testing:

salt 'hypervisor*' virt.init centos1 2 512 image=/var/lib/libvirt/images/test_centos_test.img

You can now use the virt.query command to view the newly-prepared new VM:

salt-run virt.query

The above command will show you the data about each hypervisor and its associated virtual machines.

As soon as the new VM has completed the booting process, it should have communicated with the Salt Master. If the new VM is up and running, execute a test.ping.

QEMU Copy on Write Support

You can work with the qcow disc image format for quick image cloning. You can pass a ".qcow2" image path and the "enable qcow" flag to virt.init:

salt 'hypervisor*' virt.init centos1 2 512 image=/var/lib/libvirt/images/test_centos.qcow2 enable_qcow=True start=False

Migration of the Virtual Machines

In case you want to move virtual machines, Salt Virt offers complete assistance. You can use the libvirt state in the script if you want to make migration possible.

If you want to support migration, you must provide a few items.

Many operating systems activate firewalls when they are set up; you must open up the firewall for libvirt and kvm to cross-communicate and run migration procedures.

If you are working with hypervisors of Red Hat, you need to open port 16514.

iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 16514 -j ACCEPT

You must enable the virt:tunnel option for Salt. This parameter instructs Salt to use port 16514 and to conduct migrations over the libvirt TLS tunnel for security. Without virt:tunnel, libvirt attempts to bind to arbitrary ports when doing migrations.

If you want to turn on the virt:tunnel, you can apply the following script to the master config file.

virt:
    tunnel: True

Once you have updated the master config and restarted the master. Now, you can order the minions to update the pillar so it will recognize the change:

salt \* saltutil.refresh_modules

Finally, you can execute the routines for migration. You can migrate a VM or Virtual Machine using the Salt Virt migrate script:

salt-run virt.migrate centos <new hypervisor>

Working with the VNC Consoles

Salt Virt can also help in setting up VNC consoles, enabling the opening of remote visual consoles, even if they are not enabled by default. Pass the enable vnc=True parameter when using virt.init to create a new VM in order to set up a console for it.

The information from the virt.query script will show the VNC console port for the particular VMs.

centos
  CPU: 4
  Memory: 1048576
  State: running
  Graphics: vnc - hyper6:5900
  Disk - vda:
    Size: 4.0G
    File: /srv/salt-images/ubuntu/system.qcow2
    File Format: qcow2
  Nic - ac:de:48:98:08:77:
    Source: br0
    Type: bridge

The “Graphics: vnc - hyper6:5900” line is the most important. The firewall of the hypervisor must first allow traffic to the port indicated, in this example, 5900. Once the port is open, vncviewer can be used to quickly open the console:

vncviewer hyper6:5900

Because there is no VNC security configured by default on these ports, it is advised to keep them behind a firewall and use SSH tunnels to access these VNC interfaces. Remember that any user who visits a VNC interface can watch activities on that interface and that any user who logs in can interact with the user who is currently signed in on the virtual machine.

Also check out - Strong Number

Frequently Asked Questions

What is Jinja in DBT?

In the data build tool (dbt), SQL can be combined with Jinja. Jinja is a templating language. Your dbt project into a programming environment using Jinja for SQL. It allows you to do things that aren't usually possible in SQL. For example, with Jinja, you can use control statements in SQL.

Is Jinja2 a library?

Jinja is a library for Python. It is designed to be fast, flexible, and secure. Suppose you have worked with other text-based template languages, such as Django or Smarty. Then you will find it super easy to work with.

What is the difference between Jinja and Jinja2?

Jinja 2 has a similar syntax as Jinja 1. The difference is that around the argument list, macros now require parentheses. Also, Jinja 2 allows dynamic inheritance and dynamic includes. The old helper function rendertemplate is invalid now and included used instead.

What is Jinja salt?

Jinja is the default templating language. Jinja supports a secure, sandboxed template execution environment. It is the advantage used by Salt. Salt highly recommends the usage of jinja/jinja |yaml files.

What is Salt software used for?

SALT is also known as Systematic Analysis of Language Transcripts. It is software that standardizes the process of transcribing, eliciting, and analyzing language samples. It includes standard reports, a transcription editor, and reference databases for comparison with typical peers.

Conclusion

In this article, we have studied one of the automation engines, i.e., Salt Project. We have studied the Cloud Controller part of Salt in detail. We have tried to understand hypervisors and Virtual Machines as well. We have also seen how to migrate a VM.

We hope that this article has provided you with the help to enhance your knowledge regarding Salt Project and if you would like to learn more, check out our articles on running-salt-states and commands-in-docker-containers and salt-event-systems.

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. Enroll in our courses and refer to the mock test and problems available; take a look at the interview experiences and interview bundle for placement preparations.

Do upvote our blog to help other ninjas grow.

Merry Learning!