Basics of Managing Code with Code Manager in Puppet

Introduction

Do you know what Puppet is? Have you ever wondered how to manage code with a code manager in Puppet?

Puppet is a configuration management tool. It is used for managing the infrastructure on physical or virtual devices. It is a Ruby-based open-source software. It facilitates real-time control of complicated infrastructure. Puppet has a declarative language for describing system configuration. It is a model-driven approach that requires a basic understanding of programming. The Puppet was created by Puppet, Inc., which Luke Kanies established in 2005.

Let's dive into the article to learn about Code Manager in Puppet.

Configuring post-environment hooks puppet

Customize Code Manager configuration

To personalize your Code Manager settings, set parameters in Hiera. To alter your configuration:

Open the data/common.yaml file in your control repository.
The class puppet enterprise::master::code manager needs additional parameters. Use the format:

puppet_enterprise::master::code_manager::<PARAMETER>: <SETTING>

For example, these settings reduce the maximum time permitted for deploying a single environment and increase the default worker pool size:

puppet_enterprise::master::code_manager::deploy_pool_size: 4
puppet_enterprise::master::code_manager::timeouts_deploy: 300

Run Puppet on the central server. Updates to the Code Manager configuration file are made by Puppet.

Configuring post-environment hooks

After an environment has been deployed, post-environment hooks can start custom actions.

You can configure the post-environment hook parameter in Hiera to run a list of hooks after Code Manager delivers code to an environment. An array of hashes containing the URL and use-client-ssl keys is accepted for this parameter.

A POST request should be sent to the HTTPS URL specified by the URL key. The request comprises a JSON-formatted body with details about the deployment of the environment, including:

{
     "deploy-signature":"482f8d3adc76b5197306c5d4c8aa32aa8315694b",
     "file-sync":{
       "environment-commit":"6939889b679fdb1449545c44f26aa06174d25c21",
       "code-commit":"ce5f7158615759151f77391c7b2b8b497aaebce1"},
     "environment":"production",
     "id":3,
     "status":"complete"
   }

The use-client-ssl key is a Boolean value that determines whether HTTPS connections are made using the client's SSL settings. If the server at the end of the URL uses the Puppet CA, set this to true. Set this to false if not. If nothing is specified, false is the default.

Configuring garbage collection

Environment deployments are stored in memory for an hour by default in Code Manager. Configuring waste collection will allow you to change this.

Use the deploy TTL parameter in Hiera to set the Code Manager trash collection frequency. A string with one of the following suffixes is acceptable for this parameter:

Days: d
Hours: h
Minutes: m
Seconds: s
Milliseconds: ms

For example, deployments are kept in memory by Code Manager for 30 days when deploy ttl: '30d' is specified. 1h is the default setting (one hour).

Configuring module deployment scope

Code Manager deploys modules' code incrementally by default. Modifying the deployment scope of the module code is possible using the full deploy parameter.

Only sync modules with definitions that (in the environment's Puppetfile) permit their version to "float" (like Git branches), and modules with definitions that have been added or modified since the environment's previous deployment are included in incremental deployments. Incremental deployments do not support SVN modules.

By providing the following argument to true, you can turn off incremental deploys and deploy all module code regardless of modification or float status:

puppet_enterprise::master::code_manager::full_deploy

The entire deploy parameter should be adjusted to false to enable incremental deployments once more.

Code Manager parameters

r10k-specific parameters

Triggering Code Manager on the command line

When deploying your environments from the command line, use the puppet-code command to start Code Manager.

Installing and configuring puppet-code

The puppet-code command is automatically installed and set up on your primary server by Puppet Enterprise (PE), a component of the provided PE client tools package. Additionally, you can configure the puppet-code on a workstation or agent node, alter the parameters for various users, or configure the system globally.

A JSON file located at the following address contains the global configuration options for systems running *nix and macOS:

/etc/puppetlabs/client-tools/puppet-code.conf

This configuration file includes the following by default:

{
"cacert": "/etc/puppetlabs/puppet/ssl/certs/ca.pem",
"token-file": "~/.puppetlabs/token",
"service-url": "https://<PRIMARY_HOSTNAME>:8170/code-manager"
}

The global configuration settings for Windows computers can be found at: C:\ProgramData\PuppetLabs\client-tools\puppet-code.conf

In the Windows default configuration file, you'll find the following:

{
"cacert": "C:\\ProgramData\\PuppetLabs\\puppet\\etc\\ssl\\certs\\ca.pem",
"token-file": "C:\\Users\\<username>\\.puppetlabs\\token",
"service-url": "https://<PRIMARY_HOSTNAME>:8170/code-manager"
 }

Deploying environments with puppet-code

To start a Code Manager code deployment, use puppet-code deploy. The --all flag or the name of a specific environment must be supplied (to deploy all environments). For example:

puppet-code deploy production
puppet-code deploy --all

The default puppet-code deploy ENVIRONMENT OPTION> command deploys the provided environment(s) and only returns deployment queuing results if no other parameters are specified.

Running puppet-code on Windows

You must provide the complete path to the command when running puppet-code on a Windows workstation, whether it is managed or not. For example:

C:\Program Files\Puppet Labs\Client\bin\puppet code deploy production --wait

Return deployment results (--wait)

Add the --wait flag if you want puppet-code deploy to report the actual deployment event(s) outcomes. If not, the command only delivers deployment queuing information. For example:

puppet-code deploy --all --wait

With the --wait flag, File Sync publishes code to the live code directory and all compilers before returning results after the code has been deployed to the chosen environment(s) by Code Manager.

The deployment signature, the commit SHA from the control repo used to deploy the environment, is included in the resulting message. The report also contains two additional SHAs, proving that file sync is aware that the environment has just been moved to the code staging directory.

Advanced puppet-code configuration

The puppet-code command can be set up on agent nodes, workstations not under PE management, and for specific users (on any machine).

Configure puppet-code on agents and workstations

Install the client tools package and set up the puppet-code on the computer in question if you want to use puppet-code on an agent node or a workstation that PE does not manage.

Make a configuration file called puppet-code.conf in the client tools directory on the agent node or workstation.
- The default client tools directory for Linux and Mac OS X systems is /etc/puppetlabs/client-tools.
- The default client tool directory for Windows is C:\ProgramData\PuppetLabs\client-tools.
Edit the puppet-code.conf file as necessary to change the cacert, token-file, and service-url settings if PE is not managing this machine. These have to be formatted correctly in JSON.

Configure puppet-code for different users

You can set up the puppet-code settings for specific users on any system.

In the client tools directory for the user, create a puppet-code.conf file.
- Put the file in the user's /.puppetlabs/client-tools/ directory on Linux or Mac OS X platforms.
- Put the file in the default user configuration file directory for Windows computers at C:\Users\<username>\.puppetlabs\ssl\certs\ca.pem.
Customize the cacert, token-file, and service-url settings in the user's puppet-code.conf file. These have to be formatted correctly in JSON.

puppet-code command

Options, actions, and deploy action options are all accepted by the puppet-code command. Change the puppet-code command using the format shown below:

puppet-code [GLOBAL_OPTIONS] <ACTION> [ACTION_OPTIONS]

These global settings are supported by the puppet-code command.

Global puppet-code options

These global options are supported by puppet-code.

Triggering Code Manager with a webhook

You can use a web endpoint to launch Code Manager and deploy your code by calling it with a custom script or webhook. The most straightforward approach to activating Code Manager is using webhooks.

Webhooks are supported by Code Manager for Team Foundation Server, GitHub, Bitbucket Server (formerly Stash), and Bitbucket. The control repository alone must use the webhook. Other repositories are not permitted to use it.

Create a Code Manager webhook

You must first construct a custom URL and then configure the webhook with your Git host to set up the webhook to launch environment deployments.

Create a custom URL

To enable communication between your Git server and Code Manager, create a unique URL. Code Manager supports webhooks for GitHub, Bitbucket Server (formerly Stash), Bitbucket, GitLab (just Push events), and Team Foundation Server (TFS). These components must be used in the custom webhook URL:

The simplest valid webhook URL format is:

https://<NAME>:<PORT>/code-manager/v1/webhook?type=<TYPE>

Unless you disabled authenticate webhook, the token parameter is necessary. The following is a valid URL format with the token parameter:

https://<NAME>:<PORT>/code-manager/v1/webhook?type=<TYPE>&token=<TOKEN>

The following URL structure is acceptable if your source configuration calls for the prefix parameter:

https://<NAME>:<PORT>/code-manager/v1/webhook?type=<TYPE>&prefix=<PREFIX>&token=<TOKEN>

The URLs for a GitHub webhook and a Bitbucket Server webhook, respectively, are as follows:

https://puppet.example.com:8170/code-manager/v1/webhook?type=github&token=<TOKEN>
https://puppet.example.com:8170/code-manager/v1/webhook?type=bitbucket-server&prefix=dev&token=<TOKEN>

Code Manager webhook query parameters

With the Code Manager webhook URL, you can use these query parameters.

Set up the Code Manager webhook on your Git host

Put your customized URL in the payload field of the webhook form on your Git server. JSON is the supported content type for Code Manager webhooks. Depending on your Git host, there are several procedures for setting up a webhook. For example, choose Settings > Webhooks & services on a GitHub repository, specify the payload URL, and select application/json as the content type.

You've completed setting up Code Manager once you've configured your webhook. The webhook will now activate Code Manager to deploy your code whenever you commit new code and publish it to your control repository. You can also use the POST /v1/webhook endpoint to activate your webhook manually.

Troubleshooting a Code Manager webhook

Each service type's POST /v1/webhook endpoint is reached by deployments sparked by a webhook in Stash/Bitbucket, GitLab, or GitHub.

The length of the authentication token prevents Code Manager's webhook authentication from working if you are using a GitLab version earlier than 8.5.0. Update GitLab or disable authentication to utilize the webhook with it. If you disable it, webhook authentication can only be turned off for the POST /v1/webhook endpoint. The GET /v1/deploys/status and POST /v1/deploys endpoints cannot be used without authentication.

Follow the directions for Triggering Code Manager with a Webhook and keep an eye on the Puppet Server log to diagnose webhook problems. Open another terminal window and execute the following commands to check the logs:

tail -f /var/log/puppetlabs/puppetserver/puppetserver.log

When you activate the webhook, check for errors and informational messages in the log. These errors only occur in the puppetserver.log file. If you cannot identify the issue with your webhook using this method, manually deploy to the POST /v1/deploys endpoint while keeping an eye on the console-services.log file, as explained in Troubleshooting Code Manager.

Triggering Code Manager with custom scripts

Custom scripts are a good alternative if you can't employ webhooks to start deployments. For example, if you already use continuous integration techniques, have privately hosted Git repositories, or provide notifications. A webhook makes it easier to activate Code Manager than a bespoke script. We advise using a webhook if you can.

The puppet-code command or a curl command that accesses the Code Manager API endpoints can generate a script that instructs Code Manager to deploy your environments. If at all possible, we advise using the puppet-code command.

Run your script to instruct Code Manager to deploy your new code after you've pushed it to your control repository. The guidelines that follow assume that Code Manager has been installed.

Scripting the puppet-code command

Environments can be deployed from the command line using the puppet-code command. Customized Code Manager scripts can make use of this command. Code Manager scripts use puppet-code functions similarly to those that use the command line to trigger Code Manager. The advantage of using a script is that you don't have to recreate commands from the start each time because commands are stored in scripts.

You are using the appropriate action, environment (or —all for all environments), and customized puppet-code.conf settings, and other options, to construct the puppet-code command you want. After that, add the command to your entire script.

Run your script to cause Code Manager to deploy your code after pushing new code to your control repo.

Scripting deploys curl commands.

Code deployments may be started using the Code Manager API deploy endpoint. To access this endpoint, use the curl command in your customized scripts. A scripted call to the deployed endpoint is similar to making a single call to the endpoint. You can avoid starting from scratch every time by storing commands in scripts.

As stated in POST /v1/deploys, construct your desired curl command using the pertinent URI path, parameters, and authentication. After that, add the command to your complete script.

Scripting deploys/status curl commands

A deployment's status can be checked using the Code Manager API deploys/status endpoint. To access this endpoint, use the curl command in your scripts. The deploys/status endpoint call from a script is similar to making a single call to the endpoint. You can avoid starting from scratch every time by storing commands in scripts.

As stated in GET /v1/deploys/status, construct your desired curl command using the pertinent URI path, parameters, and authentication. After that, add the command to your entire script.

Troubleshooting Code Manager

The file sync service, r10k, and source control must work together harmoniously for Code Manager to function. Check that these parts are working if Code Manager is giving you trouble.

Code Manager Logs

The Puppet Server log receives Code Manager's logs. This log is located at:

/var/log/puppetlabs/puppetserver/puppetserver.log

Check Code Manager’s Status

If your deployments aren't going as planned or if you need to make sure Code Manager is enabled before executing a dependent command, check the status of File Sync and Code Manager.

Code Manager and file sync's responsiveness can be confirmed with the puppet-code status command. The command returns similar data as by the GET /v1/deploys/status endpoint. The command, by default, returns information at the info log level. The critical and debug log levels are not supported.

Puppet-code status may report the following errors:

Error 1: Cod Manager couldn’t connect to the server

Error 2: Code Manager reports invalid configuration

Error 3: File sync storage service reports unknown status

Test the connection to the control repository

The control repository regulates the environment's existence and ensures that all required modules are installed in your environments in the correct versions. The control repository must be accessible to the primary server and cloned using the pe-puppet user.

Run the following command to verify that Code Manager can connect to the control repo:

puppet-code deploy --dry-run

This command returns a list of all environments in the control repository if the connection has been properly configured (or repos if you have multiple sources configured). A successful response indicates that the pe-puppet user may carry out the required operations, the Git URL is accurate, and the SSH key for the control repository has the proper permissions.

The command returns the following message if the connection fails:

Unable to determine current branches for Git source

The primary server's file path is also returned, which you may use to troubleshoot the SSH key and Git URL.

Check the Puppetfile for errors

Verify that the Puppetfile for the environment is free of syntax mistakes and that each module can be installed from the source specified. To achieve this, you need a duplicate Puppetfile for an environment in a temporary directory. Make a copy of the Puppetfile and put it in a temporary directory on the central server, located at /var/tmp/test-puppetfile. After that, you can check your Puppetfile's syntax and sources.

Run r10k puppetfile check from the temporary directory to verify the Puppetfile syntax. If syntax errors are found, fix them and retake the test. The command responds with Syntax OK if the syntax is accurate. Run a test installation to verify the sources' setup in your Puppetfile.

Run a deployment test

To confirm your Puppetfile syntax, source access, and whether the deployment works through r10k, you can manually execute a full r10k deployment.

Based on the r10k.yaml file used by Code Manager, the following script tries to deploy r10k in its entirety. This test does not cause file synchronization; it merely writes to the code staging directory. Use this only for ad-hoc testing. The command for test deployment is:

sudo -H -u pe-puppet bash -c \
'/opt/puppetlabs/puppet/bin/r10k deploy environment -c \
/opt/puppetlabs/server/data/code-manager/r10k.yaml -p -v debug'

If the command is successful, directory-based environments and all required modules for each environment are added to the /etc/puppetlabs/code-staging directory. If the command doesn't work, Code Manager's r10k-related settings are probably the issue. Which settings are not working is made clear by the error messages.

Code Manager API

Without having direct shell access, you can utilize the Code Manager API to deploy code and monitor the progress of deployments on your central server and compilers.

Forming Code Manager API request

The Code Manager API requires authentication and accepts properly formatted HTTPS requests. Requests must contain a URI path with the following format:

https://<DNS>:8170/code-manager/v1/<ENDPOINT>

The components of the variable route come from:

DNS: The DNS name of your central server. All options are using localhost, manually entering the DNS name or a puppet command.
ENDPOINT: A section or sections that specify an endpoint, such as deployments, webhook, or deploys/status.

For example, any of the following URLs might be used to access the GET /v1/deploys/status endpoint:

GET https://$(puppet config print server):8170/code-manager/v1/deploys/status
GET https://localhost:8170/code-manager/v1/deploys/status
GET https://puppet.example.dns:8170/code-manager/v1/deploys/status

Curl arguments, authentication, and maybe the content type and other parameters specific to the endpoint you are calling must all be included in a complete curl command.

Code Manager API authentication

Token-based authentication is necessary for Code Manager API queries. Token-based authentication or Requesting an authentication token for deployments are the places to go in PE for information on how to create, configure, revoke, and delete authentication tokens.

When using the puppet-access display command to provide tokens for deployed endpoints, you can provide an X-authentication header like:

auth_header="X-Authentication: $(puppet-access show)"
uri="https://$(puppet config print server):8170/code-manager/v1/deploys/status"
curl --insecure --header "$auth_header" GET "$uri"

Alternatively, you can use the actual token as follows:

auth_header="X-Authentication: <TOKEN>"
uri="https://$(puppet config print server):8170/code-manager/v1/deploys/status"
curl --insecure --header "$auth_header" GET "$uri"

POST/v1/deploys

Use the dry-run parameter to verify your control repo connection or trigger Code Manager to deploy code to one or more environments.

Request format

The content type for Code Manager API queries made to this endpoint is application/json. The keys listed in the following table must be used to create a JSON object for the body. Deploy-all or environments must be supplied; though not necessary, you might find the other keys helpful occasionally.

Response format

Each object in a successful response contains information about a deployed or queued environment. The following table lists the keys that are used by response objects. The value of the wait key in the request determines which keys are present and the status key's potential values.

POST/v1/webhook

By activating your Code Manager webhook, you can deploy code.

Request format

Contrary to regular POST calls, you cannot make a Code Manager API request to the webhook endpoint using a JSON body; you must include parameters to the URI path. Included in the parameters are the following:

type: always necessary to establish your Git host.
prefix: A must if prefixing is used in your source configuration. Establishes the prefix to be applied when changing branch names to environment names.
token: Necessary unless authenticate webhook has been disabled in your Code Manager setup. The authentication token must be supplied in the token parameter. Tokens that are provided as query parameters may show up in access logs.

This request, for example, contains the type and token parameters:

GET https://$(puppet config print server):8170/code-manager/v1/webhook?type=github&token=<TOKEN>

Response format

All responses are displayed in the interface of your Git provider when a push to the control repository manually triggers your Code Manager webhook. Automatic webhook triggers do not get command-line responses from Code Manager.

If you manually launch the webhook using a curl command and your request is valid and well-formed, Code Manager responds with an OK. This merely shows that the request was correct; it says nothing about whether the resulting code's deployment was successful.

Error responses

Check your Git provider interface for error messages if the webhook is automatically triggered and an error occurs. The API produces an unrecognized-webhook-type error along with a copy of the given type value and a list of valid type values if you manually trigger the webhook using a curl command and the request has a missing or incorrect type.

GET /v1/deploys/status

For each environment, learn the status of the code deployments Code Manager is handling right now. If you want to know the status of a specific deployment, you can specify an id query parameter.

Request format

Code Manager API calls made to this endpoint are simple calls with authentication, like:

type_header='Content-Type: application/json'
auth_header="X-Authentication: $(puppet-access show)"
uri="https://$(puppet config print server):8170/code-manager/v1/deploys/status"
curl --header "$type_header" --header "$auth_header" GET "$uri"

All deployments in the queue's status are returned by a simple request. By calling the position of a particular deployment in the current deployment queue, you can obtain the status of that deployment by appending the optional id query argument. According to this request, the fifth deployment in the current queue has the following status:

type_header='Content-Type: application/json'
auth_header="X-Authentication: $(puppet-access show)"
uri="https://$(puppet config print server):8170/code-manager/v1/deploys/status?id=5"
curl --header "$type_header" --header "$auth_header" GET "$uri"

Response format

The successful response for requests without the id parameter is a JSON object outlining the status of each deployment in the queue. Three secondary objects that report data on deployments based on the statuses of the deployments are divided into the body.

deploys-status: There are four arrays in the deploys-status variable that indicate the potential states of code deployment: new, queued, deploying, or failed.
file-sync-storage-status: Contains a deployed object that shows environments that Code Manager has successfully deployed to the code staging directory but has not yet synced to the live code directory. The response contains the environment name (environment), deployment date and time (date), and commit SHA from the control repository that Code Manager utilized to deploy the environment's code.
file-sync-client-status: Indicates the current state of your primary server and each compiler to which Code Manager is distributing environments. It also indicates if the staging server's code has been synchronized with the live code directory.

The following keys are used in the condensed response for requests with the id argument to characterize the requested deployment:

environment: the designation of the setting.

status: The state of the deployment at the moment of the response. Syncing, deploying, failing, new, queued or synchronized.

queued-at: the time and date the deployment was placed in a queue.

Check out most important Git Interview Questions here.

Frequently Asked Questions

How does a code manager work?

Go to the Code Manager (click on the codes icon in the left-hand menu). The window containing more specific information about the code you want to edit will open when you click on it. Click the "Add to group" button. To establish a new code group, enter its name in the box and press the "+" button.

What command should I use to deploy Puppet code?

Use the puppet-code deploy command with the environment(s) you want to deploy environments. Run puppet-code deploy with the environment specified on the command line to deploy environments. Indicate the environment by name. Use the —all flag to deploy across all environments.

What is an application code manager?

By using highly reusable libraries, the engineering tool known as Application Code Manager promotes more effective project development. Using Library Designer, you can make your libraries or use ones already there.

What is a puppet control repo?

The files and scripts used to deploy and administer Puppet environments are kept under version control in a control repository. To automate the administration and deployment of your Puppet code, you must use r10k or Puppet Enterprise Code Manager.

Conclusion

In this article, we have extensively discussed the Code Manager in Puppet. We have also explained how to customize code manager configuration, trigger code manager on the command line, trigger code manager with a webhook, trigger code manager with custom scripts, troubleshoot code manager, and more.

We hope this blog has helped you enhance your SSL and Certificate Puppet knowledge. If you want to learn more, check out our articles on Ansible vs. Puppet, DevOp's best things, DevOps tools, and reasons to build a career in DevOps.

Practice makes a man perfect. To practice and improve yourself in the interview, you can check out Top 100 SQL problems, Interview experience, Coding interview questions, and the Ultimate guide path for interviews.

Do upvote our blog to help other ninjas grow. Happy Coding!