Overview of Pillar in Salt

Introduction

Salt is a configuration management and distributed remote execution system. It was created with the goal to improve, simplify, and further customize the best remote execution solutions. It can maintain the remote nodes in defined states and query and execute commands on various nodes.

In this blog, we will discuss and learn the concept of Pillars in salt.

Pillars

Pillars are salt interfaces designed to provide global values to salt minions. The pillar’s data is managed in the same way as it is managed in the Salt state tree. Pillar is helpful for storing sensitive data to a particular minion.

Let’s see how to declare the master pillar in salt.

Declaring Master Pillar

The salt master has a pillar_roots file that matches the file_roots file saved in the file server. The pillar_roots option, like file_roots, maps environments to folders. Then, using matches in a top file that is organized similarly to the state top file, the pillar data is mapped to minions. The pillars can use the same matcher types as top files.

In the below example, we can see that the pillar_roots file is configured the same way as file_roots.

pillar_roots:
  base:
    - /srv/pillar

Let’s see the structure of the top file. 

/srv/pillar/top.sls

base:
  '*':
    - packages

In the above top file, we can see that it is declared that in the base environment, the glob matching all minions will have access to the pillar data located in the packages pillar. 

We can ignore errors related to missing pillar files with the help of the ignore_missing keyword. The example using the ignore_missing keyword is given below:

base:
  '*':
    - servers
    - systems
    - ignore_missing: True

Let’s see what is there in the packages pillar. 

/srv/pillar/packages.sls

{% if grains['os'] == 'Debian' %}
apache: apache2
git: git-core
{% elif grains['os'] == 'RedHat' %}
apache: httpd
git: git
{% endif %}

In the above pillar, the pillar sets two key-value pairs. If the minion is running Debian, then the apache key is set to apache2, and the git key is set to git-core. Otherwise, if the minion is running RedHat, then the  apache key is set to httpd, and the git key is set to git. 

Let's see how to create a dynamic pillar environment.

Dynamic Pillar Environments

 To use dynamic git branch based environments for pillar files, make sure that the environment __env__ is specified in pillar_roots. Let’s see an example. 

pillar_roots:
  __env__:
    - /srv/pillar

ext_pillar:
  - git:
    - __env__ https://example.com/git-pillar.git

Let’s see how to do namespace flattening in Pillar.

Namespace Flattening in Pillar

During the namespace flattening, all the .sls files are merged into a single dictionary of key-value pairs. There could be an error if the same key is defined in multiple .sls files. 

Let’s take a look at the example below.

In top.sls, 

base:
  '*':
    - packages
    - services

The packages.sls contains the key-value pairs bind: bind9. 

And the services.sls contains the key-value pairs bind: named. 

In the above example, the request for the bind pillar key will return named only. The bind9 value will be lost because of services.sls pillar was evaluated later. 

To solve the above problem, its recommended to structure the pillar files with more hierarchy. For example, the packages.sls file can also be configured like this:-

packages:
  bind: bind9

Let’s see how to do dictionary merging in the pillar.

Dictionary Merging in Pillar

The dictionary merging is required when the same pillar key is used in multiple .sls files, and the keys in all the files refer to nested dictionaries. In this case, the content from these dictionaries is merged recursively.

Let's take an example where the top.sls file will remain the same, and other SLS files are modified:-

packages.sls

bind:
  package-name: bind9
  version: 9.9.5

services.sls:

bind:
  port: 53
  listen-on: any

The dictionary for the above example will look like this:

$ salt-call pillar.get bind
local:
    ----------
    listen-on:
        any
    package-name:
        bind9
    port:
        53
    version:
        9.9.5

Let’s learn about In-memory pillar data.

In-Memory Pillar Data

It is very computationally expensive to compile all the pillar data. To avoid this problem, the salt minions maintain a copy of pillar data in memory and send a copy of the pillar data each time pillar data is requested. 

The functions such as pillar.item, pillar.get, and pillar.raw returns the in-memory pillar data. 

If you are writing your custom execution modules, then the in-memory pillar data is available as the __pillar__ dunder dictionary. 

You can refresh the in-memory pillar data using the following command:-

salt '*' saltutil.refresh_pillar

Let’s see how to handle the pillar environment.

How to handle Pillar Environment?

If there are multiple pillar environment that is running simultaneously. By default, all the pillar data from all the environments are merged together, and the pillar dictionary will contain the keys from all the configured environments. 

The pillarenv minion config option is used to force minions to take configuration from a single environment. This can be helpful for running states with different pillar data, whether for testing or quality assurance purposes or to test changes to the pillar data before releasing them online. 

For example, in the minion config file, the pillarenv is set to base.  

Note: In running states, the pillarenv CLI argument can be used to override the minion's pillarenv config value.

To run a pillar in the testing phase, run the following command.

salt '*' state.apply mystates pillarenv=testing

To run a pillar in dev mode, run the following command.

salt '*' state.apply mystates saltenv=dev

or

salt '*' state.apply mystates saltenv=dev pillarenv=dev

To run a pillar in QA mode, run the following command.

salt '*' state.apply mystates saltenv=dev pillarenv=qa

Let’s learn how to view pillar data.

Viewing Pillar Data

We can use the pillar execution module in order to view pillar data. The pillar execution module has several functions, each having its own significance.

The execution functions are:-

• pillar.item: Its function is to fetch the value of one or more keys from the in-memory pillar data.
   
• pillar.items: Its function is to compile a fresh pillar dictionary and return it when required.
   
• pillar.raw: Its function is quite similar to pillar.items, it also returns the entire pillar dictionary but from the in-memory pillar data, not from compiled fresh pillar dictionary.
   
• pillar.get: Its function is to return the demanded data.
   

Let’s see how to set pillar data using the command line.

Setting Pillar data using CLI

The below example clearly describes how to set pillar data using CLI.

salt '*' state.apply pillar='{"coding": "ninjas"}'

This will add a pillar key of coding with its value set to ninjas.

Note: Avoid sending any sensitive data using CLI.