Do you think IIT Guwahati certified course can help you in your career?
Introduction⛳
You can automate and manage server setup using a tool called puppet. Puppet is composed of various packages, which are collectively known as the puppet platform. You organize, save, and execute your puppet code using this platform. You can also use hiera in puppet.
In this article, we will study about hiera in puppet.
👉Usage of Puppet Code
Modules are used to store puppet code. Use the tested and pre-built modules on Puppet Forge, a repository of hundreds of modules created by Puppet developers and the Puppet community. Modules control particular infrastructure operations like installing and configuring applications. Code and data are both found in modules. What enables you to alter your configuration is the data.
You can isolate the data from the code and store it in one place using a tool named Hiera. This enables you to establish known parameters and variants, guardrails, and parameters so that your code is thoroughly tested and you can evaluate all the parameter edge cases.
👉Designing System Configurations in Puppet
Designing system configurations in puppet typically involves managing all of the software,services, and configuration, necessary for a user on a specific system.
➡️The Roles and Profiles Method
The roles and profiles technique can make the code reusable, reconfigurable, and refactorable. It also helps to keep complexity under control.
➡️Roles and Profiles Example
This example shows the complete roles and profiles workflow. Use it to comprehend the roles and profiles approach in its entirety. By modifying this example code to a greater level of complexity, other examples demonstrate how to create complicated configurations.
➡️Designing Advanced Profiles
We gradually redesign or refactor our basic roles and profiles to address real-world issues. With a few slight modifications, the end outcome is the Jenkins profile that Puppet uses in production.
➡️Designing Convenient Roles
There are various methods for creating roles, and you must choose the one that works best for you and your team. Start with specific roles and only move from them in small, careful stages.
👉About Hiera in Puppet
Hiera in Puppet is for two purposes:
Key-value pairs should be used to store the configuration data.
Look up the information a specific module requires for a particular node while compiling the catalog.
This is achieved by:
Auto Parameter searching for classes in the catalog
Calls for explicit lookups
The "defaults, with overrides" pattern is used by Hiera's hierarchical lookups, allowing you to declare common data just once and override it when the default doesn't function. Hiera specifies data sources using Puppet facts, allowing you to organize overrides to match your infrastructure. Although employing facts for this purpose is typical, data sources can also be defined without using facts.
🔶Hiera Hierarchies
Hiera searches for information using a hierarchy, an ordered collection of data sources.
A hiera.yaml configuration file is used to set up hierarchies. The hierarchy instructs Hiera on accessing a specific data source at each level. Typically, a hierarchy is structured as follows:
---
version: 5
defaults: # used for any hierarchy level that leaves out these keys.
datadir: data # The hiera.yaml directory is relative to this path.
data_hash: yaml_data # Use the built-in YAML backend.
hierarchy:
- name: "Per-node data" # Human-readable name.
path: "nodes/%{trusted.certname}.yaml" # File path, relative to datadir.
# ^^^ IMPORTANT: include the file extension!
- name: "Per-datacenter business group data" # Uses custom facts.
path: "location/%{facts.whereami}/%{facts.group}.yaml"
- name: "Global business group data"
path: "groups/%{facts.group}.yaml"
- name: "Per-datacenter secret data (encrypted)"
lookup_key: eyaml_lookup_key # Uses non-default backend.
path: "secrets/%{facts.whereami}.eyaml"
options:
pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
- name: "Per-OS defaults"
path: "os/%{facts.os.family}.yaml"
- name: "Common data"
path: "common.yaml"
💠Hierarchies Interpolate Variables
In most hierarchical levels, variables are integrated into their configuration:
path: "os/%{facts.os.family}.yaml"
Variable syntax with percent and brackets is a Hiera interpolation token. It is comparable to the $expression interpolation tokens used in the Puppet language. Wherever an interpolation token is used, Hiera calculates the variable's value and adds it to the hierarchy.
The facts.os.family accesses members of hashes and arrays using the Hiera unique key.subkey notation. In the Puppet language, it is equivalent to $facts['os']['family'], but the 'dot' notation generates an empty string rather than issuing an error if some data is missing. Be careful to avoid having an empty interpolation match an unwanted path.
💠Hiera Searches the Hierarchy in Order
Hiera verifies the concrete data sources in the order they were written after replacing the variables to create a list of them. In general, Hiera moves on to the next source until it finds one that does exist; then, it uses it if the data source doesn't exist or doesn't specify a value for the current key. The usual merging approach may not always apply; for instance, Hiera can combine the data from all sources and use it.
Priority is given to earlier data sources over newer ones. The node-specific data in the example above has the highest priority and can take precedence over data at any other level.
Business group data is divided between local and global sources, with the local source taking precedence over the latter. All nodes always send common data last.
💠Layered Hierarchies
Each layer of the data used by Hiera has its own hiera.yaml file.
Every layer can set up an autonomous hierarchy. Hiera combines them into a single super-hierarchy: global → environment → module.
Consider the environment hierarchy in the previous example. If the global hierarchy also included the following:
---
version: 5
hierarchy:
- name: "Data exported from our old self-service config tool"
path: "selfserve/%{trusted.certname}.json"
data_hash: json_data
datadir: data
Additionally, the default data hierarchy for the NTP module was as follows:
Hiera uses three distinct configuration levels. Before doing a lookup, each layer's hierarchy is linked into a single super-hierarchy. The three layers are searched in the following sequence: global → environment → module.
💠The Global Layer
The default location of the global layer's configuration file is $confdir/hiera.yaml. By altering the hiera_config setting in puppet.conf, you can modify the location. One global hierarchy exists in Hiera. Since it comes before the environment layer, it can be used for short-term overrides, such as when your operations team wants to skip the usual change processes.
When migrating your backends to support Hiera 5, it's crucial to keep in mind that the global layer is the only area where legacy Hiera 3 backends can be used. It is compatible with hiera.yaml v5, hiera.yaml v3(deprecated), and other configuration formats.
💠The Environmental Layer
By default, the <ENVIRONMENT DIR>/hiera.yaml directory is where the environment layer's configuration file can be found. Most of your Hiera data hierarchy definition takes place at the environment layer. Each Puppet environment has a unique hierarchy configuration that applies to all of the environment's nodes. These configuration formats are supported: v5 and v3 (deprecated).
💠The Module Layer
By default, a module's MODULE>/hiera.yaml contains the configuration file for a module layer. The class parameters of a module are given default values and merge behavior by the module layer. It is a practical replacement for the params.pp pattern.
The environment data specified by a user overrides the default data set by the module's creator because it comes last in Hiera's lookup order.
Each module is capable of having a unique hierarchical configuration. Only keys in the module's namespace can have data bound to them. For instance:
Lookup Key
Relevant Module Hierarchy
ntp::servers
ntp
jenkins::port
jenkins
secure_server
(none)
👉Getting Started with Hiera
This section explains the fundamental ideas and actions you need to get started using Hiera, such as how to write data and create a hiera.yaml configuration file.
⭕Create a hiera.yaml Config File
Hiera's configuration file's name is hiera.yaml and for each environment should be separate.
Make a new file called hiera.yaml in one of your environments' main directories. Add the following content by pasting it there:
# <ENVIRONMENT>/hiera.yaml
---
version: 5
hierarchy:
- name: "Per-node data" # A name that is readable by people.
path: "nodes/%{trusted.certname}.yaml" # Relative file path to datadir.
# ^^^ IMPORTANT: include the file extension!
- name: "Per-OS defaults"
path: "os/%{facts.os.family}.yaml"
- name: "Common data"
path: "common.yaml"
The result is that this file is in a format called YAML, which is used extensively throughout Hiera.
⭕The Hierarchy
An ordered list of data sources is configured as a hierarchy in the hiera.yaml file.
In the order that they are written, Hiera searches these data sources. Lower-priority sources take precedence over higher-priority ones. Most hierarchy levels use variables to identify a data source, resulting in various nodes receiving diverse data.
Hiera's fundamental idea is a defaults-with-overrides pattern for data lookup, employing a list of data sources specific to each node.
⭕Write data: Create a Test Class
When applying the catalog, a test class creates a temporary file on the agent and writes the data it receives there.
The first step is to develop a Puppet class for testing since Hiera is used with Puppet code.
1️⃣Create a module called profile if you haven't already done so to implement the roles and profiles technique. Profiles are wrapper classes that configure a layered technology stack using some component modules.
2️⃣Create a class called hiera_test.pp using your profile module's Puppet Development Kit (PDK).
3️⃣To your hiera test.pp file, add the following code:
The test class requests configuration information via class parameters. Using the lookup key CLASS NAME>::PARAMETER NAME>, Puppet searches Hiera for class parameters.
4️⃣Create a manifest with the class included:
# site.pp
include profile::hiera_test
5️⃣Compile the catalog and note that it fails because some values are necessary.
6️⃣Set these keys in your Hiera data to provide the missing class parameters and their values. You can include the settings in your common, OS, or per-node data, depending on where in your hierarchy you wish to put them.
7️⃣Recompile and note that the parameters are now found automatically.
Parameter
Hiera key
$ssl
profile::hiera_test::ssl
$backups_enabled
profile::hiera_test::backups_enabled
$site_alias
profile::hiera_test::site_alias
⭕Write Data: Set Values in Common Data
In the level at the bottom of your hierarchy, set values in your shared data.
This hierarchy level stores the data in a YAML file using the YAML backend. The following details should be put together to determine where to store that file:
✔️The directory of the current setting.
✔️A subdirectory of the environment is the data directory. It is by default <ENVIRONMENT>/data.
✔️File path determined by hierarchy level.
In this case, /etc/puppetlabs/code/environments/production/ + data/ + common.yaml.
Set values for two of the class's parameters by opening that YAML file in an editor.
You can exclude the third argument, $site_alias, from the data because the default value for it is defined in the code.
⭕Write Data: Set Per-Operating System Data
The second level of the hierarchy uses the os fact to find its data file. According to the present node's operating system, it can employ a variety of data files.
Assume that your developers use MacBook notebooks with the Darwin OS family for this example. Set $backups_enabled to false to prevent data from being sent to your production backup server when a developer is running an app instance on their laptop.
If you don't have any Mac laptops running Puppet, pick an OS family that matters to your infrastructure.
Follow the below steps to write data by setting per-operating system data.
1️⃣Find the data file by entering the value you want to target in place of %{facts.os.family}.
/etc/puppetlabs/code/environments/production/data + os/ + Darwin +.yaml
3️⃣Compile to verify that the override is implemented.
⭕Write Data: Set Per-Node Data
You can set data by name for each node because the top level of the example hierarchy uses the value of $trusted['certname'] to identify its data file. In this example, we'll assume you have a server called jenkins-prod-03.example.com, set up to serve this application at the hostname ci.example.com and configured to use SSL. Pick the name of a real server you can use to test this out.
1️⃣Replace %{trusted.certname} with the name of the node you're aiming for to find the data file.
3️⃣Compile to verify that the override is implemented.
⭕Testing Hiera Data on the Command Line
It is crucial to double-check the data a node receives as you set Hiera data or restructure your hierarchy. The puppet lookup command facilitates interactive data testing. For example:
puppet lookup profile::hiera_test::backups_enabled --environment production --node jenkins-prod-03.example.com
This gives back the word true. To efficiently employ the puppet lookup command:
✔️Run the command on a node connected to your Puppet Server or a node with access to a complete copy of your Puppet code and configuration.
✔️The node you are testing against should have made at least one communication with the server so that the lookup command can access its information (otherwise you need to supply the facts yourself on the command line).
✔️To provide the command access to your live data, ensure that the command uses the global confdir and codedir. Include –codedir and –confdir in the command line if you're not executing puppet lookup as root user.
✔️When using PuppetDB, you can use --node to use any node's facts in a lookup by specifying –node<NAME>. Hiera can automatically obtain the actual facts for that node to resolve data.
✔️Provide facts in a YAML or JSON file and include that file as part of the command with the --facts<FILE> if you do not use PuppetDB or wish to test for a set of facts that do not exist. Run facter -p –json > facts.json to obtain a file containing all the facts rather than create one from scratch. Copy the facts.json file to your Puppet Server node, make any necessary changes, and then paste it from a node similar to the one you wish to analyse.
For several systems, the Puppet Development Kit includes preconfigured fact sets. If you wish to test against platforms you don't have or if you want "typical facts" for a particular platform, you can use those.
✔️Try rerunning the command with –explain if you are not receiving the values you anticipate. Hiera outputs a thorough explanation of the data sources it examined and the results it discovered with the –explain flag.
👉Configuring Hiera
Hiera.yaml is the name of the Hiera configuration file. It configures the hierarchy for a certain layer of data. This section explains about configuring hiera.
🔶Location of hiera.yaml Files
In a typical deployment, numerous hiera.yaml files are included. The module and environment layers of Hiera's configuration hierarchy generally contain several instances.
The locations of each layer's configuration files are:
The following order is followed by Hiera's data search: global➡ environment➡ module.
🔶Config file syntax
There is a hash with up to four top-level keys in the YAML file hiera.yaml. In the hiera.yaml file, the following keys are found:
✔️version - Essential. It must be 5 without any quotation marks.
✔️defaults: A hash that can be used to establish a default datadir, backend, and hierarchy level parameters.
✔️hierarchy: An array of hashes used to set the hierarchy's tiers.
✔️default_hierarchy: It is an array of hashes that establishes a default hierarchy that will only be used if the standard hierarchy entries are unsuccessful. Only acceptable in a module's hiera.yaml file.
version: 5
defaults: # Any hierarchy level that doesn't include these keys is used.
datadir: data # The hiera.yaml directory is relative to this path.
data_hash: yaml_data # Use the built-in YAML backend.
hierarchy:
- name: "Per-node data" # Human-readable name.
path: "nodes/%{trusted.certname}.yaml" # Relative file path to datadir.
# ^^^ IMPORTANT: include the file extension!
- name: "Per-datacenter business group data" # Uses custom facts.
path: "location/%{facts.whereami}/%{facts.group}.yaml"
- name: "Global business group data"
path: "groups/%{facts.group}.yaml"
- name: "Per-datacenter secret data (encrypted)"
lookup_key: eyaml_lookup_key # Uses a non-default backend.
path: "secrets/nodes/%{trusted.certname}.eyaml"
options:
pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
- name: "Per-OS defaults"
path: "os/%{facts.os.family}.yaml"
- name: "Common data"
path: "common.yaml"
💠The Default Configuration
version: 5
hierarchy:
- name: Common
path: common.yaml
defaults:
data_hash: yaml_data
datadir: data
Only if the file is present and specifies version: 5 are these defaults applied. It disables Hiera for that layer if hiera.yaml is missing. Different defaults apply if it specifies a different version.
💠The Default Keys
You can omit the lookup function and datadir keys from your hierarchy levels by setting default values for those keys using the defaults key. A hash with the keys datadir, options, and one of the mutually exclusive lookup function keys must be the value of defaults. A hash can have up to three keys.
Any file-based hierarchy level that doesn't define its datadir will use datadir as the default value. The datadir is the directory containing the data in the same directory as the hiera.yaml configuration file if it is not specified.
options: a generic value for options applied to any hierarchy level without a specific value.
Any hierarchy level that doesn't define its lookup function keys is used. That must be one of:
✔️data_hash : Key-value pairs are hashed by the data hash (typically from a data file)
✔️lookup_key: generates values one key at a time (typically for a custom data provider)
✔️data_dig: Key by key values are produced using data dig (for a more advanced data provider)
✔️hiera3_backend: An old Hiera 3 backend is called out by the data provider hiera3 backend (global layer only).
The YAML, JSON, and HOCON built-in data providers have a key called data hash and a value called yaml_data, json_data, or hocon_data respectively.
💠The Hierarchy key
The hierarchy key sets the hierarchy's tiers. An array of hashes must be the value of the hierarchy. The hash's keys should be four spaces indented to align with the first key. To visually separate hashes, place an empty line between them. For instance:
hierarchy:
- name: "Per-node data"
path: "nodes/%{trusted.certname}.yaml"
- name: "Per-datacenter business group data"
path: "location/%{facts.whereami}/%{facts.group}.yaml"
💠The default_hierarchy Key
A top-level key is the default hierarchy. When and only when the ordinary hierarchy's lookup fails to yield a value, it is started. The usual merging rules are applicable inside this default hierarchy. Environment or global layers cannot contain the default hierarchy.
The values found in the default_hierarchy have no impact on the values found in the regular hierarchy if lookup_options are used, and vice versa. The lookup function only uses the merge parameter in the typical hierarchy. It has no impact on how a value is put together in the default_hierarchy. Use of the lookup_options found in the default hierarchy is the only way to change that.
🔶Configuring a Hierarchy Level: Built-in Backends
YAML, JSON, and HOCON are the three built-in backends for Hiera. These all use files as their data sources. These backends can be used in any hierarchy, and you can also mix them with unique backends. Set default settings for the datadir and data_hash keys if most of your data is in a single file type.
The following keys are required for each hierarchy level of YAML/JSON/HOCON:
✔️name — This level's name is displayed in debug messages and --explain output.
✔️The data files for this hierarchy level are path, paths, glob, globs, or mapped_paths (choose one).
These paths require a file extension, permit variable interpolation, and are relative to the datadir.
mapped_paths do not support glob expansion.
✔️data_hash: specifies which backend to use. It can be omitted if a default is set. One of the following must apply to the value:
For YAML, use yaml_data.
For JSON, use json_data
For HOCON, use hocon_data.
✔️datadir: Data files are saved in the datadir directory. If the default is set, it may be omitted.
The entire path to the data directory is /etc/puppetlabs/code/environments/production/data if the config file is located at /etc/puppetlabs/code/environments/production/hiera.yaml and the datadir is set to data.
The datadir can be set to an absolute path in the global layer, but it must always be relative to the other layers.
Interpolating variables is supported by datadir.
💠Specifying File Paths
Key
Data Type
Expected Value
path
String
One file path.
paths
Array
There are countless file paths. When many files are present, Hiera searches each one in the order they were written, acting as if it were a sub hierarchy.
glob
String
One glob pattern akin to a shell that might match any number of files. If more than one file is discovered, Hiera scans them all in alphabetical order.
globs
Array
numerous shell-like glob designs are possible. If more than one file is discovered, Hiera scans them all in alphabetical order (ignoring the order of the globs).
mapped_paths
Array or Hash
A fact that consists of an assortment (or hash) of values. These values are expanded by Hiera to create an array of paths.
🔶Configuring a Hierarchy Level: hiera-eyaml
A native interface for the Hiera eyaml extension is included in Hiera 5 (Puppet 4.9.3 and later), which retains data encrypted on a disc but permits Puppet to read it during catalog building. The eyaml backend in hiera.yaml is similar to the conventional built-in backends, with a few exceptions: it uses a lookup_key rather than data_hash, and it needs an options key to find the decryption keys. It should be noted that the eyaml backend can read both plain-text and encrypted yaml files.
These keys are required for each level of the eyaml hierarchy:
✔️name — This level's name is displayed in debug messages and ---explain output.
✔️lookup_key: Which backend should be used? eyaml_lookup_key must be the value and should be used instead of using the data hash setting.
✔️The data files for this hierarchy level are path, paths, mapped paths, glob, or globs (choose one). These paths require a file extension, permit variable interpolation, and are relative to the datadir. Normally, you'll use .eyaml in this situation. They function the same way they do with more common backends.
✔️datadir: Data files are saved in the datadir directory. If the default is set, it may be omitted. The same principles apply as they do to ordinary backends.
✔️options: A hash of hiera-eyaml-specific options primarily used to set up decryption. The following keys must be present in this hash for the default encryption algorithm:
➡pkcs7_private_key: PKCS7 private key location specified by pkcs7 private key.
➡pkcs7_public_key: public key location specified by pkcs7 public key.
➡If you use a different encryption plugin, look up the encryption options in the plugin's
documentation. To replace the pkcs7 options, set an encrypt_method option and certain plugin-specific parameters.
➡In this hash, regular strings may be used as keys; symbols are unnecessary.
The options and file path keys support variable interpolation.
🔶Configuring a Hierarchy Level: Legacy Hiera 3 Backends
You can employ specialized data backends created for Hiera 3 in your global hierarchy if you rely on them. The environment and module layers do not support them. These keys are required for each level of the legacy hierarchy:
✔️name — This level's name is displayed in debug messages and —explain output.
✔️path or paths — The data files for this hierarchy level are path or paths.
⏹️Even though you would have left it out in the v3 hiera.yaml file, you still need to
include the file extension for backends that work with files.
⏹️Don't use a file extension for backends that don't support files.
✔️hiera3_backend — is the recommended legacy backend. You would enter this name in the :backends key of the config file for version 3.
✔️datadir: Data files are saved in the datadir directory. Only set this if your backend's backend-specific options demand the a:datadir setting.
⏹️The entire path to the data directory is /etc/puppetlabs/code/environments/production/data if the config file is located at /etc/puppetlabs/code/environments/production/hiera.yaml and the datadir is set to data. Remember that Hiera v3 uses "hieradata" rather than "data"
⏹️You can set the datadir in the global layer to an absolute path.
✔️options — A hash containing any backend-specific settings (apart from datadir) that your backend requires. This would have been in a top-level key named after the backend in the v3 configuration. Regular strings are acceptable as keys. They are changed into symbols for the backend by Hiera.
Using legacy backends, the example below illustrates pretty much identical hiera.yaml files for versions 3 and 5:
Hiera supports custom backends. The following keys are required for the hash that represents each hierarchy level:
✔️name — This level's name is displayed in debug messages and ---explain output.
✔️A backend key, one of which must be:
⏹️data_hash
⏹️lookup_key
⏹️data_dig - A more specialized version of lookup_key called data dig is appropriate when the backend is a database. Unlike lookup_key, data_dig resolves dot-separated keys.
⏹️hiera3_backend - (global layer only)
✔️a path or URI (Universal Resource Identifier) key - if the backend so requests. These keys support variable interpolation. There are provided the following path/URI keys:
⏹️path
⏹️paths
⏹️mapped_paths
⏹️glob
⏹️globs
⏹️uri
⏹️uris- These URIs, often known as paths, function similarly for built-in backends. Any backend that supports a path automatically supports paths, glob, and globs because Hiera handles the task of finding files. Any data source type can be represented by uri (string) and uris (array). Before calling the backend, Hiera does not check if URIs can be resolved, and it is not required to comprehend any specific URI schema. A backend can only use the options key to locate its data and omit the path/URI key entirely.
✔️datadir — The location of the data files: The path points to the directory containing hiera.yaml. Only necessary if the backend uses the path(s) and glob(s) keys; optional if a default is set.
✔️options — A hash of additional options for the backend, such as database login information or where to find a decryption key. Variable interpolation is supported for all values in the options hash.
The function's name that implements the backend API must be the value of any key you use. Remember that the decision is made here by the person who implemented the specific backend, not the user.
👉Creating and Editing Data
This section discusses merge behaviour and interpolation, two crucial features of using Hiera.
🔶Set the Merge Behavior For a Lookup
In Hiera, various values for a given key are frequently found in distinct data sources. By default, Hiera only returns the first value it discovers, but it has the option to keep looking and combine all the values it finds.
✔️There are two ways to configure the merge behavior for a lookup:
⏹️At lookup time: Although it supports the lookup method, automated class parameter lookup is not supported.
⏹️Using the lookup_options key in Hiera data. Both manual and automatic lookups can use this. Additionally, it enables module authors to define default actions that users can override.
✔️Both of these methods allow you to define a merge behavior as a string, such as 'first,' or a hash, such as { "strategy" => "first"}. The hash syntax can be used with different merging types and deep merges (which provide more options).
🔶Merge Behaviors
You can pick between the first, unique, hash, and deep merge behaviors. One of the following identifiers should be used when describing a merging behavior:
⏹️'knockout_prefix' - string or undef; disabled by default.
⏹️'sort_merged_arrays' - Boolean; default is false
⏹️'merge_hash_arrays' - Boolean; default is false
💠First
The first value for the key is returned in a first-found lookup, and the remaining values are ignored. This is how Hiera usually acts.
Specify this merge behavior with one of these:
'first'
{'strategy' => 'first'}
lookup($key)
Nothing (because it’s the default)
💠Unique
To create a merged, flattened array with all matching values for a key, a unique merge, also known as an array merge, combines any number of array and scalar (string, number, boolean) values. All redundant values are eliminated. If any of the values are hashes, the lookup fails. From highest priority to lowest priority, the outcome is arranged.
Specify this merge behavior with one of these:
'unique'
lookup($key, { 'merge' => 'unique' })
{'strategy' => 'unique'}
💠Hash
To create a merged hash of all keys that match their corresponding values, a hash merge combines the keys and values of any number of hashes. If any of the values are not hashes, then none of the matches are valid, and the lookup fails. Hiera uses the value from the data source with the highest priority when multiple source hashes with the same key exist; it does not recursively merge the values.
Puppet hashes maintain the sequential sequence of its keys. Hiera starts with the data source with the lowest priority when merging hashes. It changes the current keys in place and adds new keys to the end of the hash for each source with a higher priority.
{
a => "common value",
b => "per-node override", # Use the higher-priority source's value while maintaining the #lower-priority source's order.
c => "other common value",
d => "per-node value",
}
Specify this merge behavior with one of these:
'hash'
lookup($key, { 'merge' => 'hash' })
{'strategy' => 'hash'}
💠Deep
A deep merge creates a merged hash by combining the keys and values of any number of hashes. It is a compact External Node Classifier(ENC) with an array of class names. Hiera recursively merges the source hashes if the same key appears in more than one of them:
✔️Another deep merge is used to combine hash values.
✔️Merged array values are used. This is distinct from the singular merge. The result is sorted in the opposite direction of the unique merge, from lowest priority to highest. The outcome is not flattened and can therefore include nested arrays. The result may also alter based on the merge_hash_arrays, and sort_merged_arrays settings.
✔️In a first-found lookup, the highest priority value is used for scalar (String, Number, and Boolean) variables.
Specify this merge behavior with one of these:
'deep'
include(lookup($key, { 'merge' => 'deep' }))
{'strategy' => 'deep', <OPTION> => <VALUE>, ...} — Adjust the merge behavior with additional options.
🔶Set Merge Behavior at Lookup Time
To override previously defined merging behavior for a key, use merge behavior at lookup time. To specify a merge behavior as a parameter or flag, use the lookup function or the puppet lookup command.
Function example:
# Merge several arrays of class names into one array:
lookup('classes', {merge => 'unique'})
Command line example:
$ puppet lookup classes --merge unique --environment production --explain
🔶Set lookup_options to Refine the Result of a Lookup
You can choose merge behavior and use the convert_to key to receive automatic type conversion when setting lookup_options, which helps you further filter the results of a lookup.
💠The lookup_options Format
The value of lookup_options is a hash. It follows the following format:
lookup_options:
<NAME or REGEXP>:
merge: <MERGE BEHAVIOR>
💠Location for Setting lookup_options
The default merging behavior for other keys in your data can be changed by setting the lookup options metadata keys in Hiera data sources, including module data. In any lookup that doesn't specifically override a key's configured merge behavior, Hiera uses that behavior.
💠Defining Merge Behavior lookup_options
Set the lookup options key in your Hiera data source to configure merge behavior:
# <ENVIRONMENT>/data/common.yaml
lookup_options:
ntp::servers: # Name of key
merge: unique # Merge behavior as a string
"^profile::(.*)::users$": # Regexp: `$users` parameter of any profile class
merge: # Merge behavior as a hash
strategy: deep
merge_hash_arrays: true
💠Overriding Merge Behavior
A hash merge is carried out when lookup options are given to Hiera. For specific keys, higher-priority sources take precedence over lower-priority lookup options. A default merging behavior for a specific key can be configured in a module, and users of that module can define overrides in the environment layer. The following setup, as an illustration, specifies lookup_options for several keys in a module. The environment can override one of the keys, but the others keep their configuration.
# <MYMODULE>/data/common.yaml
lookup_options:
mymodule::key1:
merge:
strategy: deep
merge_hash_arrays: true
mymodule::key2:
merge: deep
mymodule::key3:
merge: deep
# <ENVIRONMENT>/data/common.yaml
lookup_options:
mymodule::key1:
merge: deep # this overrides the merge_hash_arrays true
💠Overriding Merge Behavior in a Call to lookup()
The configured merge behavior is overridden when a merge behavior is specified as an argument to the lookup function. For instance, with the setup described:
lookup('mymodule::key1', 'strategy' => 'first')
The lookup_options setting uses strategy 'first' rather than 'deep' for the lookup of 'mymodule::key1'.
💠Make Hiera Return Data by Casting to a Specific Data Type
Use the lookup options key convert to, which accepts either a type name or an array of type names and arguments, to convert values from Hiera backends to rich data values that are not representable in YAML or JSON.
You get automatic type-checking with convert_to. For instance, an error is raised if you specify a convert_to the type "Enum['red', 'blue', 'green']". The value searched up is not one of those strings. When the Puppet code performing the lookup does not provide adequate type checking, you can use this to assert the type.
Specify the data type in string form for types with single-value constructors, such as Integer, String, Sensitive, or Timestamp.
For example, to turn a String value into an Integer:
Specify the type and the arguments in an array if the constructor needs them. Additionally, if a data type constructor accepts optional parameters, you can define it in this manner.
For instance, to translate the string "042" into an integer with a base ten detailed interpretation of the string:
In lookup_options, you can simultaneously set merging behavior for multiple lookup keys by using regular expressions.
The merge behavior for profile::server::users, profile::postgresql::users, and profile::jenkins::server::users is set by a regular expression key like '^profile::(.*)::users$'. Based on Ruby's regular expressions, Puppet's regular expression support is used for regular expression lookup options. To use a regular expression in the lookup_options function:
✔️Create a quoted string using the pattern. Avoid using the forward-slash (/.../) regular expression delimiters from Puppet.
✔️The start-of-line metacharacter (^, commonly known as a carat) should be used to start the pattern. Hiera uses it as a literal key name rather than a regular expression if ^ isn't the first character.
✔️If this data source is in a module, add ^ and the module's namespace, composed of the module's complete name and the::namespace separator, after. For instance, the ntp module requires that each regular expression lookup option begins with ^ntp::. Any other approach will lead to mistakes.
All lookup keys that match that pattern are subject to the merge behavior you specify for that pattern. Hiera resolves conflicts when several lookup alternatives may be applicable for a given key. For instance, Hiera uses a literal option when one is provided (rather than a regular expression). Otherwise, Hiera applies the first regular expression, in the order they occur in the module code, that matches the lookup key.
🔶Interpolation
Using the syntax %{variable} in Hiera, you can interpolate or insert a variable's value into a string. Hiera employs interpolation twice:
The path, paths, glob, globs, uri, uris, datadir, mapped_paths, and options of a hierarchy level can all be interpolated with variables. This enables every node to receive a unique hierarchy.
Data: Interpolation can be used to prevent a recurrence. This can be done in one of two ways:
⏹️Refer to the fact directly rather than manually transcribing it if a value always involves the value of a fact (for instance, if you need to specify a mail server and have one mail server with a predictable name per domain).
⏹️Write out the value for one key and reuse it for the others using the lookup or alias interpolation functions if many keys need to share the same value. As a result, updating data is simpler because you only need to make a single modification to a specific value.
🔶Interpolate a Puppet Variable
The value of a Puppet top scope variable is the most typical thing to interpolate. The most practical and predictable variables for Hiera are the facts hash, trusted hash, and server_facts hash. Avoid using namespace variables from classes, local variables, Hiera-specific pseudo-variables, and namespaced variables from classes unless the class has already been evaluated(pseudo-variables are not supported in Hiera 5).
See Puppet variables given to Hiera if you are using the pseudo-variables in Hiera 3. Facts can be accessed by puppet in two different ways: collectively in the facts hash ($facts['networking']), and separately as top-scope variables ($networking).
When using distinct fact variables, be sure to give them the (empty) top-scope namespace, as in:
%{::networking} ✅
%{networking} ❌
A Puppet variable can be interpolated using its name without the leading dollar sign ($). When attempting to access a data structure member, use the Hiera key.subkey notation. To interpolate the value of $facts['networking']['domain’] for instance “smtpserver: "mail.%{facts.networking.domain}" should be written.
🔶Interpolation Functions
Interpolation functions can be used in Hiera data sources to insert non-variable values. These are not the same as Puppet functions; only Hiera interpolation tokens support them. Five interpolation methods exist:
✔️lookup - interpolates the value from a key's search in Hiera into a string.
✔️hiera- hiera is another word meaning lookup.
✔️alias- a key is looked up in Hiera using the alias function, and the value is used to replace the surrounding string. If the value is exactly one alias, the result has the same data type as the aliased key and is not converted to a string.
✔️literal - a method for writing the percent sign (%) literally without inadvertently interpolating anything.
✔️scope- An alternative method of interpolating a variable is scope. generally ineffective.
💠The lookup and hiera Function
The lookup and hiera interpolation functions return a value after looking up a key. Any other outcome will result in an error. The lookup must return a string. The hiera interpolation functions retrieve a value from a key and return it. Any other outcome will result in an error. The lookup must return a string.
Assume, for instance, that the MySQL profile requires a database server and that you have previously configured that hostname in the data. You can type:
# in location/pdx.yaml:
profile::mysql::public_hostname: db-server-01.pdx.example.com
# in location/bfs.yaml:
profile::mysql::public_hostname: db-server-06.belfast.example.com
# in common.yaml:
profile::wordpress::database_server: "%{lookup('profile::mysql::public_hostname')}"
💠The alias Function
You can reuse Hash, Array, Boolean, Integer,or String values by using the alias function. When you interpolate an alias in a string, Hiera completely replaces that string with the aliased value while maintaining the original data type. For instance:
The literal interpolation function allows you to escape a literal percent sign (%) in Hiera data to prevent interpolation from occurring where it is not desired. This is helpful, for instance, when working with Apache configuration files that contain text such as %{SERVER_NAME}. For instance:
The scope interpolation function interpolates variables. It functions in the same way as variable interpolation. The function's argument is a variable's name. The following two values would be the same:
This section explains how Puppet looks up data with Hiera for class parameter values.
🔶The Puppet lookup Function
The lookup function uses Hiera to find a value for a given key. The lookup function fails compilation by default if no values are available and return the first value discovered. The lookup function can also be set up to combine many values into one. Hiera examines up to four hierarchy layers of data when looking up a key in the following order:
Global hierarchy.
The hierarchy of the current environment.
If the key has the format <MODULE NAME>::<SOMETHING>, then the indicated module's hierarchy will be displayed.
The lookup is repeated if steps 1-3 did not provide a value and the module's hierarchy contains a default_hierarchy element in its hiera.yaml file if not found.
🔶The Puppet lookup Command
The puppet lookup command is the command line interface (CLI) for Puppet's lookup feature. Using the puppet lookup command, you can perform Hiera lookup from the command line. It must be executed on a node that contains a copy of your Hiera data. You can use sudo to manage puppet lookup after logging onto a Puppet Server node.
You can test and explore your data by using the puppet lookup command, which searches your Hiera data and provides a value for the provided lookup key. The hiera command has been replaced. Hiera uses a node's facts to determine where to get the relevant data sources. puppet lookup by default uses information from the node you execute the command on, but by using the –nodeNAME option, you can obtain information for any other node. The lookup command, if feasible, uses the actual stored facts from PuppetDB for the given node. Pass facts from a JSON or YAML file using the --facts FILE option to pass facts if PuppetDB is not configured or if you want to offer additional fact values.
For example:
Puppet Server node’s facts can be used to look up key_name:
$ puppet lookup key_name
🔶Access Hash and Array elements Using a key.subkey Notation
Hiera's key.subkey notation can be used to access hash and array members. When doing the following actions, you can access hash and array elements:
Variable interpolation into hiera.yaml or a data file. Deeply nested data structures make up many of the most widely used variables, such as facts and trusted.
Using the lookup function or the puppet lookup command. Look up a single member of a hash or array using lookup('some_key.subkey'), if the value of lookup('some_key') is a hash or array.
Using interpolation methods that perform Hiera lookups, such as lookup and alias.
🔶Hiera Dotted Notation
Only literal keys are recognized in the Hiera dotted notation; arbitrary expressions are not supported. Literal dots can be included in a hash's key text. A hash holding all certificate extensions for a node, for instance, is the value of $trusted['extensions'], although some of its keys can be raw OID strings like "1.3.6.1.4.1.34380.1.2.1." The key.subkey notation in Hiera can be used to access those values, but you must enclose the affected subkey in single or double quotation marks. Use the opposite type of quote for the subkey and escape quotes (as required by your data file format) to prevent prematurely terminating the entire string if the compound as the real key is quoted (for instance, as needed for the lookup interpolation function).
The term "backend" refers to a special Puppet function that receives a specific set of parameters and whose return value adheres to a specific format. To find its data, the function is free to take any action. There are differences in the performance characteristics of various forms of data. Three different backend kinds, namely data_hash, lookup_key, and data_dig, are supported by Hiera to ensure that it functions effectively with all types of data sources.
🔶data_hash Backends
A complete data source is read at once by a data_hash backend function, which then delivers the contents as a hash. The built-in backends for YAML, JSON, and HOCON are all data_hash operations. The function must either invoke the not found method of the context object or return a hash of the lookup keys and their corresponding values. The hash could be null.
Example signature in puppet language:
function mymodule::hiera_backend(
Hash $options,
Puppet::LookupContext $context,
)
🔶lookup_key Backends
A single key is looked up, and its value is returned by a backend function called the lookup_key. For instance, the lookup key function in the hiera_eyaml backend is built-in. The function must either return a value for the requested key or call the not_found method of the context object. It may give a value of undef.
Example signature in puppet language:
function mymodule::hiera_backend(
Variant[String, Numeric] $key,
Hash $options,
Puppet::LookupContext $context,
)
🔶data_dig Backends
In contrast to a lookup_key function, which looks up a single key, a data_dig backend function looks up a single set of keys and subkeys.
Using key.subkey notation, Hiera enables you to look for specific elements of hash and array values. Use data_dig types in cases where:
✔️Lookups are relatively costly.
✔️The data source can separate elements from the hash and array values.
✔️The lookup function receives the key.subkey requests to access portions of big data structures.
The function must call the context object’s not_found method or return a value for the requested sequence of key segments.
Example signature in puppet language:
function mymodule::hiera_backend(
Array[Variant[String, Numeric]] $segments,
Hash $options,
Puppet::LookupContext $context,
)
🔶Hiera Calling Conventions for Backend Functions
For each distinct key lookup, Hiera calls the data_hash function once per data source, the lookup_key functions once per data source, and the data_dig functions once per data source for each distinct sequence of key segments.
However, depending on the choices for the path, paths, uri, uris, glob, and globs, a given hierarchy level may refer to several data sources. Each hierarchy level is handled by Hiera as follows:
✔️When using the path, paths, glob, or globs settings, Hiera detects which files are present and makes a separate call to the function for each one. The function will not be invoked if no files are discovered.
✔️When using the uri or uris options, Hiera makes one call to the function for each URI.
✔️Hiera only calls the function once if none of the options are used.
If the inputs change, Hiera can invoke a function for a specific data source again. When Hiera.yaml interpolates a local variable into a file path, it executes the function again for scopes where the variable's value has changed. You must interpolate only facts, trustworthy facts, and server facts in the hierarchy because doing otherwise severely impacts performance.
🔶The Options Hash
The choices The following keys can be found in a hash, depending on whether path, glob, uri, or mapped_paths have been set:
✔️path: The complete disc location of a file. It is present only when path, paths, glob, globs, or mapped_paths are present in the hierarchy. Without the file, Hiera never calls the function.
✔️uri: A uri that your function can use to find a data source. Only when uri or uris are present in the hierarchy is it current. Before sending the URI to the function, Hiera does not validate it.
✔️Every key from the options is set for the hierarchy level. List any options your backend requires or accepts. The path and uri keys are reserved.
👉Upgrading To Hiera 5
Hiera 5 upgrade has various significant benefits. A real environment data layer makes it easier to use multiple backends in your hierarchy, create a custom backend, and make changes to your hierarchy routine and testable.
The built-in data formats used by Hiera 3 are also used by Hiera 5.
🔶Considerations For hiera-eyaml Users
For Hiera 5, we included a built-in hiera-eyaml backend in Puppet 4.9.3. This implies that you can transfer your current encrypted YAML data into the environment layer together with your other data.
🔶Considerations For Custom Backend Users
Await the release of new backends. However, upgrading will be more difficult if you continue to use custom Hiera 3 backends because you can't migrate legacy data to the environment layer until a Hiera 5 backend is available.
🔶Considerations For Custom data_binding_terminus Users
The puppet.conf file has an obsolete data_binding_terminus setting that modifies how automated class parameter search operates. Hiera (the default), none (deprecated; disables auto-lookup), or the name of a custom plug-in can be used to specify this option. The outcomes of automatic lookups for the same keys change from function-based lookups when a custom data_binding_terminus is used.
It's safe to upgrade your configuration to Hiera 5 if you're one of the few people who use this functionality because you've already had to adapt your Puppet code to prevent that issue. It would help if you replace your custom terminus with a Hiera 5 backend because we deprecated that extension point. Integrate your Hiera 5 backend into your hierarchies, after which you should remove the data_binding_terminus setting.
Per-environment hierarchy configuration is one of Hiera 5's core features. Local hiera.yaml files are more logical and practical than one global hierarchy because you most likely store data in each environment.
🔶Enable the Environment Layer for Existing Hiera Data
The environment layer can be gradually enabled. The legacy Hiera functions transition to Hiera 5 mode in migrated environments, allowing them to access environment and module data without requiring code modifications.
In each environment:
✔️Verify your code for calls to Hiera functions that contain "hierarchy override parameters", which lead to mistakes.
✔️Add a local Hiera.yaml file.
✔️If you have any custom backends, update them.
✔️To keep this environment from being included in the global layer, rename the data directory. The global layer, verified before the environment layer, is still used by unmigrated environments. Choose a separate data directory name for migrated environments if you wish to keep both migrated and unmigrated environments during the migration process.
Since it is also the new default, the new name "data" is a wise decision. There is no set time limit or downtime associated with this process. The global hierarchy can gradually be eliminated or drastically reduced once all your environments have been transferred.
✔️If the optional hierarchy overrides an argument passed to any of the classic Hiera functions (hiera, hiera array, hiera hash, and hiera include) in your Puppet code, remove it.
The hierarchy override parameter in Hiera 5 is incorrect. Search for calls with two or more commas to quickly locate instances of this argument being used. Search your codebase using the following regular expression:
This causes some false positives but aids in identifying flaws before the code is executed.
Alternately, move on to the next phase and correct issues as they arise. If you migrate a temporary branch of your control repo first, then point some canary nodes at it to ensure everything works as expected, you're probably using environments for code testing and promotion.
When you execute your canary nodes, you'll be proving your belief that you've never used hierarchy override arguments. If you discover any issues, you can correct them before merging your branch to production, just like you would with any code still under development.
✔️For the following two stages, pick a different name for the data directory. In Hiera 3, the default data directory name was <ENVIRONMENT>/hieradata, while in Hiera 5, the default is <ENVIRONMENT>/data. Use the new default if you previously used the old one. If you were already using data, make a different decision.
✔️Add a Hiera 5 hiera.yaml file to the environment. A Hiera configuration file that interacts with the environment's current data is required. See moving a version 3 hiera.yaml to version 5 if this is your first environment. Don't forget to mention the new datadir name. Copy the hiera.yaml file from a migrated environment and make any necessary adjustments if you've already migrated to at least one environment.
The generated file should be saved as <ENVIRONMENT>/hiera.yaml. /etc/puppetlabs/code/environments/production/hiera.yaml, as an example.
✔️Install any custom backends that have been ported to Hiera 5 that your data depends on in the environment. As Puppet modules, Hiera 5 backends are distributed, allowing any environment to use a unique copy of them.
✔️Move the environment's data directory by renaming it from its previous name (hieradata) to its new name if you only use file-based Hiera 5 backends (data). The global layer still requires access to its data if you use a custom file-based Hiera 3 backend; thus, you must sort the files: Hiera 3 data remains in the old data directory while Hiera 5 data is moved to it. Move the remaining files to the new datadir after your custom backends have Hiera 5 versions. If you employ non-file backends without a data directory:
Decide that configuring this data in the global hierarchy is the best option and leave it indefinitely.
Make a new database table for moved data and move values into it as you switch environments. This is equal to moving the datadir.
Till the migration is finished, permit the global and environment layers to use the same configurations for this data.
✔️For each setting, repeat these steps. Most of your environments can be migrated using the merging features in your version control system if you manage your code by mapping environments to branches in a control repository.
✔️Delete the sections of your global hierarchy that you converted into environment hierarchies after migrating the environments with active node populations.
🔶Convert a Version 3 hiera.yaml to Version 5
The hiera.yaml file is supported in three different versions by Hiera 5: versions 3, 4, and 5. If you have been using Hiera 3, the hiera.yaml file at the global layer currently represents your setup.
Translation of a version 3 configuration to a version 5 involves two migration tasks:
✔️New v5 hiera.yaml files being made for environments.
✔️Your global configuration has to be updated to accommodate Hiera 5 backends.
Even though the global hierarchy has some unique capabilities, these two processes are essentially identical.
You have to use Ruby symbols as keys in Hiera 3. Short sequences that begin with a colon are symbols, such as hierarchy. Although symbols won't (yet) result in problems, you can use ordinary strings as keys in the version 5 configuration format. The leading colons on keys can be removed.
Discard any settings that are no longer needed. Remove everything from this example except for the setting of the hierarchy.
The following settings should be deleted as they are no longer required: :logger, :merge behavior, :deep merge options
The following settings should be deleted but should be copied and pasted into a temporary file for future use: :backends, :yaml or :mongodb
Add a version key, with a value of 5:
version: 5
hierarchy:
# …
Decide on a default data directory and backend.
Set a defaults key with values for datadir and one of the backend keys if most of your data is stored in one backend, such as YAML or JSON.
For Hiera 5, the backend names have been modified, and the backend configuration has been divided into three settings:
Hiera 3 Backend
Hiera 5 Backend Setting
yaml
data_hash: yaml_data
json
data_hash: json_data
eyaml
lookup_key: eyaml_lookup_key
Interpret the hierarchy. Hierarchies in versions 3 and 5 operate differently:
Hiera runs through the entire hierarchy for each backend in version 3 since hierarchy levels do not have designated backends.
Each hierarchy level in version 5 has a defined backend and its unique configuration for that backend.
Remove hierarchy levels that use the pseudo-variables calling module, calling class, and calling class path permitted in Hiera 3. Use the module data layer or the glob pattern instead of these variables for anything you did. Version 5 of Hiera.yaml does not support these. Hierarchy levels that interpolate them should be removed.
Convert built-in backends to version 5's configuration, which specifies the hierarchy as an array of hashes. Use the data hashkey to set the backend for hierarchy levels using built-in backends, such as YAML and JSON. The following keys are set:
name- a name that people can read.
path or paths: This is the path you added a file extension to in your version 3 hiera.yaml hierarchy.
data_hash- The backend to use yaml_data for YAML, json_data for JSON.
datadir: The data directory is named datadir. It is relative to the directory containing the hiera.yaml file in version 5.
You can omit data_hash and datadir if default settings have already been provided for them.
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Per-node data (yaml version)"
path: "nodes/%{trusted.certname}.yaml" # Add file extension.
# Omitting datadir and data_hash to use defaults.
- name: "Other YAML hierarchy levels"
paths: # Can specify an array of paths instead of one.
- "location/%{facts.whereami}/%{facts.group}.yaml"
- "groups/%{facts.group}.yaml"
- "os/%{facts.os.family}.yaml"
- "common.yaml"
Backends for hiera-eyaml, which function similarly to the other built-in backends, can be translated. The distinctions are:
You must install the hiera-eyamlgem and use a different backend configuration. Instead of data_hash: yaml, use lookup_key: eyaml_lookup_key. An options key with links to the public and private keys is required for each hierarchy level. This cannot have a global default.
Custom Hiera 3 backends can be translated. Use the version 3 backend in a version 5 hierarchy at the global layer if there is no update; it is incompatible with the environment layer. Use the previous value when configuring a legacy backend's backend-specific parameters. The version 3 configuration for MongoDB in the example has the following settings:
So, for a per-node MongoDB hierarchy level, write the following:
- name: "Per-node data (MongoDB version)"
path: "nodes/%{trusted.certname}" # No file extension
hiera3_backend: mongodb
options: # Change keys to plain strings using old backend-specific options.
connections:
dbname: hdata
collection: config
host: localhost
You've converted the example setup into the following v5 config by performing the steps above:
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Per-node data (yaml version)"
path: "nodes/%{trusted.certname}.yaml" # Add file extension
# Omitting datadir and data_hash to use defaults.
- name: "Per-node data (MongoDB version)"
path: "nodes/%{trusted.certname}" # No file extension
hiera3_backend: mongodb
options: # Change keys to plain strings using old backend-specific options.
connections:
dbname: hdata
collection: config
host: localhost
- name: "Per-group secrets"
path: "groups/%{facts.group}.eyaml"
lookup_key: eyaml_lookup_key
options:
pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
- name: "Other YAML hierarchy levels"
paths: # Can specify an array of paths instead of a single one.
- "location/%{facts.whereami}/%{facts.group}.yaml"
- "groups/%{facts.group}.yaml"
- "os/%{facts.os.family}.yaml"
- "common.yaml"
🔶Convert an Experimental(Version 4) hiera.yaml to Version 5
There might be version 4 hiera.yaml files in your environments and modules if you used the experimental version of Puppet lookup, which is Hiera 5. These can be used by Hiera 5, although conversion is required, especially if you intend to use a backend other than YAML or JSON. Versions 4 and 5 of a format are comparable.
Take a look at this version 4 hiera.yaml file as an example:
Make the following adjustments to upgrade to version 5:
✔️Set the version key's value to 5.
✔️Every file path should have a file extension added; use "common.yaml" rather than "common."
✔️Add a path to any hierarchy levels that are missing one. Path no longer defaults in version 5 to the value of the name.
✔️Change the top-level datadir key to a defaults key if one exists. Pick a default backend. For example:
defaults:
datadir: data
data_hash: yaml_data
✔️Remove the backend key from each hierarchy level and replace it with a data_hash key. (If a default backend was set in the defaults key, it could be omitted here.)
v4 backend
v5 equivalent
backend: yaml
data_hash: yaml_data
backend: json
data_hash: json_data
✔️Disable Puppet lookup for an environment or module by deleting the environment_data_provider and data_provider settings. These settings are in the following locations:
environment_data_provider in puppet.conf.
environment_data_provider in environment.conf.
data_provider in a module’s metadata.json.
After being converted to version 5, the example looks like this:
# /etc/puppetlabs/code/environments/production/hiera.yaml
---
version: 5
defaults:
datadir: data # Datadir has moved into `defaults`.
data_hash: yaml_data # Default backend: New feature in v5.
hierarchy:
- name: "Nodes" # Can omit `backend` if using the default.
path: "nodes/%{trusted.certname}.yaml" # Add file extension!
- name: "Exported JSON nodes"
data_hash: json_data # Specifying a non-default backend.
paths:
- "nodes/%{trusted.certname}.json"
- "insecure_nodes/%{facts.networking.fqdn}.json"
- name: "Virtualization platform"
path: "virtual/%{facts.virtual}.yaml" # Name and path are now separated.
- name: "common"
path: "common.yaml"
🔶Convert Experimental Data Provider Functions to a Hiera 5 data_hash Backend
A function that returned a hash could be created by setting data_provider = function in the experimental custom backend support of puppet lookup. You could change your function to a Hiera 5 data hash backend if you used it.
✔️Your initial function didn't accept any arguments. Change the signature so that it will now accept a Hash and an object Puppet:LookupContext.
✔️Remove the data_provider setting, which allows a module to use Puppet lookup. This setting can be found in a module's metadata.json file.
✔️For the impacted environment or module, create a version 5 hiera.yaml file and add the following hierarchy level:
- name: <ARBITRARY NAME>
data_hash: <NAME OF YOUR FUNCTION>
🔶Updated Classic Hiera Function Calls
Hiera function
Equivalent lookup call
hiera('secure_server')
lookup('secure_server')
hiera_array('ntp::servers')
lookup('ntp::servers', {merge => unique})
hiera_hash('users')
lookup('users', {merge => hash}) or lookup('users', {merge => deep})
hiera_include('classes')
lookup('classes', {merge => unique}).include
🔶Adding Hiera Data to a Module
Modules' class parameters require default values. Previously, using the "params.pp" pattern was the favored method. Instead, you can use Hiera 5's "data in modules" strategy.
💠Module Data With the params.pp Pattern
The params.pp pattern makes use of the class inheritance functionality of Puppet. Your module has one class that sets variables for the other classes. The name of this class is <MODULE>::params. This class employs conditional logic based on the target operating system and constructs values using Puppet code. The module's other classes derive from the params class. You can set the variables in the params class to default values in their parameter lists. When employing the params.pp pattern, the values defined in the params.pp defined class are only used as defaults when no values are found in Hiera. They cannot be used in lookup merges or Automatic Parameter Lookup (APL).
You can change an existing params class to a hash-based Hiera backend using Hiera 5's proprietary backend architecture. Create a Puppet function that outputs a hash in order to build a Hiera backend.
Using the params class as a starting point:
# ntp/functions/params.pp
function ntp::params(
Hash $options, # We ignore both of these arguments, but
Puppet::LookupContext $context, # the function still needs to accept them.
) {
$base_params = {
'ntp::autoupdate' => false,
# Keys must begin with the namespace of the module, in this case `ntp::`.
'ntp::service_name' => 'ntpd',
# Use key names that can be looked up automatically using class parameters.
#The `$service name` parameter of the `ntp` class corresponds to this key.
}
$os_params = case $facts['os']['family'] {
'Debian': {
{ 'ntp::service_name' => 'ntp' }
}
default: {
{}
}
}
#If this platform uses a non-standard service name, override it when merging the hashes.
$base_params + $os_params
}
When a function is ready, instruct Hiera to use it by including it in the module layer of hiera.yaml. path, datadir, and options keys are unnecessary for a simple backend like this. If you want the same behavior as the earlier parameters, you can add it to the default_hierarch. If you want the values to be merged with values of greater priority when a merging lookup is given, use the normal hierarchy and the params.pp pattern. Whether it makes sense to merge a value or not, you can divide the key-values so that some are in the hierarchy and others in the default_hierarchy.
Add it to the regular hierarchy:
# ntp/hiera.yaml
---
version: 5
hierarchy:
- name: "NTP class parameter defaults"
data_hash: "ntp::params"
# We only require one hierarchy level since all the data is provided by a single function,.
The major classes in your module can be made simpler with Hiera-based defaults:
There is no requirement that they derive from another class.
The = operator does not require you to define a default value explicitly.
Instead, APL takes effect without a value for each argument. As with other values in Hiera, the function ntp::params is used in the example to retrieve the default parameters, which may then be modified or combined.
# ntp/manifests/init.pp
class ntp (
# default values are in ntp/functions/params.pp
$autoupdate,
$service_name,
) {
...
}
💠Module Data With YAML Data Files
Basic Hiera YAML files can be used to manage the default data for your module. In your module layer hiera.yaml file, create a hierarchy:
To specify the intended state of your system, the user can use the declarative language provided by Puppet.
In documents referred to as manifests, the user will outline the ideal condition of the system. Manifests specify how the operating system and network resources should be set up, including files, packages, and services. Puppet then organizes these manifests into catalogs and applies each catalog to the associated node in the user's infrastructure to ensure the node is configured correctly.
The evaluation order is crucial to several aspects of the Puppet language. Variables, for instance, need to be set before they may be referenced. Puppet highlights places in the language reference where the sequencing of sentences is essential.
👉Puppet Man Pages
The command line tools for puppet are made up of a single puppet binary with numerous subcommands. These following subcommands form the core of Puppet’s tool set:
puppet agent - applies the client configuration retrieved from the puppet master to the local host.
puppet apply - applies a standalone Puppet manifest to the local system.
puppet cert - independent certificate issuing body. Although it can generate certificates, it is mainly used to sign certificate requests from puppet clients.
puppet master - the central puppet server which by default performs the role of a certificate authority.
puppet module - builds, sets up, and searches for modules on the Puppet Forge.
puppet resource - the resource abstraction layer shell which Uses the Puppet RAL to directly interact with the system.
puppet lookup - uses the command line to do Hiera lookups.
We hope you have learned everything about hiera in puppet. 🙌 Check out this problem - Redundant Braces
Frequently Asked Questions
What is Puppet?
With the aid of Puppet, you can manage and automate the configuration of servers. When using Puppet, specify the ideal state for the infrastructure systems you want to manage.
What is DevOps?
DevOps is a collection of cultural ideas, operational procedures, and technical resources that rapidly enhance an organization's ability to deliver products and services.
What language is Puppet built on?
Puppet is built using Ruby’s domain-specific language.
Is Puppet an open-source tool?
Puppet is an open-source tool. Puppet is also available in the enterprise version.
What are the requirements for designing system configurations in puppet
For designing system configurations in puppet, the roles and profile method, designing advanced profiles and designing convenient roles are needed.
Conclusion
In this blog, we explored about hiera in puppet and learned about the usage of puppet code. We further discussed puppet man pages and the Puppet language.
We hope this blog has helped you enhance your knowledge about hiera in puppetand if you want to learn more, check out our articles here.
19+ registered 
