Do you think IIT Guwahati certified course can help you in your career?
Introduction
Hello, Readers!!
Have you guys heard about Puppet Enterprise?
No? Don't worry. We got you covered. Puppet Enterprise, or PE, is the commercial version of Puppet. It is built on top of the Puppet platform. It allows the IT operations teams to manage and automate more infrastructure and complex workshops in a simple and powerful way.
In this blog, we will discuss about Orchestrating Puppet.
So let us begin!!
Orchestrating Puppet runs, tasks, and plans
Users can use Puppet orchestrator as a tool to make on-demand changes to their infrastructure.
Using the orchestrator, users can initiate Puppet, task, or plan runs whenever they need them, eliminating manual work across their infrastructure.
How does Puppet orchestrator work: using the Puppet orchestrator, the user can run Puppet, tasks, or plans on-demand.
Setting up orchestrator workflow: when the orchestrator is used with PE tools, it allows the user to control when and how the infrastructure changes are made before it reaches the user's production environment.
Configuring Puppet orchestrator: once PE or the client package tools are installed, there are a few tasks that the user needs to do in order to prepare their PE infrastructure for orchestration services.
Run Puppet on demand: the orchestrator can be used to run jobs from the console, command line, or even through the orchestrator API endpoints.
Tasks in PE: tasks are ad-hoc actions that a user can execute on a target and run from the command line or the console
Plans in PE: plans allow the user to tie together tasks, scripts, commands, and other plans in order to create a complex workflow. The user can install modules that contain plans or write their own.
Orchestrator API v1: it can be used to run jobs and plans on-demand, schedule tasks and plans, and get details about jobs, plans, and events.
How Puppet orchestrator works
Using the Puppet orchestrator, the user can run Puppet, tasks, or plans on-demand. When the user runs Puppet on-demand with the orchestrator, then the user controls the rollout of configuration changes as in when and how the user wants them. The user controls when the Puppet runs and where the node catalogs are applied. The user doesn't have to wait for arbitrary run times to update the nodes.
The Puppet tasks allow the user to execute actions on a target machine. Puppet plans are bundles of tasks that are combined with other logic. They allow the user to do complex operations.
Puppet orchestrator technical overview
In order to execute on-demand Puppet runs on the agent nodes in the infrastructure, the orchestrator uses pe-orchestration services. The orchestrator uses Puppet Execution Protocol agents to orchestrate changes across the infrastructure.
The orchestrator controls the functionality for the puppet jobs, puppet tasks, and puppet plan commands. It also controls the functionality for jobs as well as single-node runs in the PE console.
There are several components in the orchestrator, each component has its own configuration and log locations.
Puppet orchestrator architecture
Puppet Execution Protocol, the Puppet Communication Protocol, and the Agentless Catalog Executor Server drives the orchestrator's functionality.
Puppet Execution Protocol
It is the message format that is used to request a task to be executed on a remote host. It also receives the responses on the status of that task.
Puppet Execution Protocol agent
It is a system service in the agent package that runs Puppet Execution Protocol.
Puppet Communications Protocol
It is the underlying communication protocol that describes how Puppet Execution Protocol messages get routed to an agent and back to the orchestrator.
Puppet Communications Protocol broker
It is a JVM-based service. It runs either in pe-orchestration-services on the primary server or in the pe-puppetserver service on compilers. They route the PCP messages, which mention the message's content and identify the sender and intended recipient.
Agentless Catalog Executor service
It is a Ruby service that enables users to execute tasks and plans, and Puppet runs remotely on agentless targets.
What happens during a task run from the orchestrator?
There are several PE services which interact whenever the user Runs Puppet on demand from their orchestrator.
The user uses the puppet job command in order to create a job in the orchestrator.
The orchestrator validates the user's token with PE RBAC service.
The orchestrator requests the environment classification from the node classifier for the targeted nodes of the job. The orchestrator also queries the PuppetDB for the nodes.
The orchestrator then creates a job ID and starts polling nodes in the job in order to check their statuses.
The orchestrator queries PuppetDB regarding the agent version on the nodes in the job.
After this, the orchestrator tells the PCP broker to start the runs on the nodes in the job. Then the Puppet runs get started on those agents.
PCP broker gets the run results from the agents.
The orchestrator gets the run results. After that, the node run reports from PuppetDB are requested by the orchestrator.
What happens during the task run from the orchestrator?
During the task run, several services interact. This is because tasks are Puppet code, that needs to be deployed into an environment on the primary server. The Puppet server exposes the task's metadata to the orchestrator. When a task is running, the orchestrator sends the PXP agent a URL that indicates from where the task is to be fetched. The task file is then downloaded through the supplied URL by the PXP agent for future use. The file gets validated against the checksum before every execution. The process comprises of the below-mentioned steps:
A task command is sent by the PE client.
The orchestrator then checks if the user is authorized or not.
The node target is fetched from the PuppetDB by the orchestrator, and it then returns the list of the targeted nodes.
The orchestrator requests task data from the Puppet Server.
Task metadata, file URLs, and file SHAs are returned by the Puppet Server.
The task command is validated by the orchestrator, which then sends the job ID back to the client.
The orchestrator sends the task parameters and the file information to the PXP agent.
A provisional response gets sent to the orchestrator by the PXP agent. The PXP agent then checks the SHA against the local cache as well as requests task files from the Puppet Server.
The task file is returned to the PXP agent by the Puppet server.
The task runs.
The results are sent to the orchestrator by the PXP agent.
Events are requested from the orchestrator by the client.
The results are returned from the orchestrator to the orchestrator.
Debugging the orchestrator and related components
If the user wants to debug the orchestrator or any of the related components, then the following mentioned log locations might be useful:
Pe-orchestration-services
The main log file is located at the below-mentioned location:
The main log file for the PCP brokers on the compilers is located at the below-mentioned location:
/var/log/puppetlabs/puppetserver/pcp-broker.log
PXP agent
In the case of the PXP agent, the main log file's location depends on the OS and can be configured if required. In the case of windows, the default location is mentioned below:
The main log file for the ACE server is located at the below-mentioned location:
/var/log/puppetlabs/ace-server/ace-server.log
Configure Puppet orchestrator
Once the user has installed PE or any other client tools package, then there are a few things that the user needs to do in order to prepare their PE infrastructure for orchestration services.
The user needs to set PE RBAC permissions as well as token authentication for the Puppet orchestrator.
The user also needs to review the orchestrator configuration files as well as adjust them as per requirement.
Orchestration services settings
Global logging and SSL settings
The settings that are shared across the PE orchestration services are contained by:
No changes are required in the global.certs file. It contains the below-mentioned settings:
Allow a list of trapperkeeper services to start
The list of trapperkeeper services from the orchestrator and the PCP broker projects, which are loaded when the pe-orchestration-services system service starts, is:
In order to disable a service from the list, remove it or comment it with a # character and then restart the pe-orchestration-services
Add puppetlabs.trapperkeeper.services.nrepl.nrepl-service/nrepl-service to the list to enable an NREPL service for debugging and restart the pe-orchestration-services
Analytics trapperkeeper service configuration
The internal settings for the analytics trapperkeeper service are contained by /etc/puppetlabs/orchestration-services/conf.d/analytics.conf.
Authorization trapperkeeper service configuration
The internal settings for the authorization trapperkepper service are contained by:
The orchestrator's configuration files allow the user to run commands from the CLI without passing additional flags. There are two types of configuration files: global configuration file and user-specified configuration file, irrespective of whether the user is running the orchestrator from the primary server or from a separate workstation.
Orchestrator global configuration file
If the user is running the orchestrator from a PE-managed machine, on either the primary server or an agent node, then PE manages the global configuration file.
The file is installed on both the workstation managed and non-managed at:
The puppet_enterprise::profile::controller is the class which manages the global configuration file. The below-mentioned parameters and values are available for this class:
Orchestrator user-specified configuration file
Users can manually create a user-specified configuration file and then populate it with the orchestrator configuration file settings. The PE does not manage this file.
The file created needs to be present at the below-mentioned location:
~/.puppetlabs/client-tools/orchestrator.conf
The user-specified configuration always takes precedence over the global configuration file if it is present.
Setting PE RBAC permissions and token authentication for orchestrator
Before a user can start running any orchestrator job, the user needs to set appropriate permissions in PE RBAC and establish token-based authentication.
The below-mentioned permissions are required by most orchestrators in order to run orchestrator jobs or tasks:
Assign task permissions to a user role
Click on the User roles tab from the Access control page.
Select the user from the list of user roles that you want to have task permissions.
Select Tasks in the Type box from the Permissions tab.
Select a task from the Object list and click on Run tasks for Permission.
Click on Add permission and commit the changes.
Using token authentication
The user needs to generate an RBAC access token in order to authenticate to the orchestration service before running an orchestrator job. If the user tries to run a job without a token, then the PE prompts the user to supply the credentials.
PE Bolt server configuration
The PE Bolt server provides the user with an API for running tasks over SSH and WinRM using the Bolt, which is the technology underlying PE tasks. The server is a Puma application, which runs as a standalone service.
The below-mentioned parameters are included in the server that is configured in /etc/puppetlabs/bolt-server/conf.d/bolt-server.conf and is managed by the puppet_enterprise::profile::bolt_server class:
Frequently Asked Questions
Name any tool that can be used to make on-demand changes to the infrastructure.
The Puppet orchestrator is an effective tool that can be used to make on-demand changes to the infrastructure.
What is the role of a PCP broker?
The PCP broker routes the PCP messages that declare the content of the message and also identifies the sender and intended recipient.
What is the default value for the ssl-port setting?
The default value for the ssl-port setting is 8143.
Conclusion
In this article, we have extensively discussed the Orchestrating Puppet.
If you wish to enhance your skills in Data Structures and Algorithms, Competitive Programming, JavaScript, etc., you should check out our Guided path column at Coding Ninjas Studio. We at Coding Ninjas Studio organize many contests in which you can participate. You can also prepare for the contests and test your coding skills by giving the mock test series available. In case you have just started the learning process, and your dream is to crack major tech giants like Amazon, Microsoft, etc., then you should check out the most frequently asked problems and the interview experiences of your seniors that will help you in landing a job in your dream company.
19+ registered 
