External Logging Handlers
The salt project has a robust log-handling system. It has internal logging features and even provides third-party integration for handling logs efficiently. It offers integration with fluent, log4mongo, logstash and sentry. These are some of the industry-leading logging solutions that serve thousands of servers across the globe. The procedure is simple; we have to set up the configuration at the salt master or minion and then define the type of logs and log levels. That's all; we are up and running with our logging system!!
Fluent Logging Handler
Fluentd is an open-source data collector and has a unified logging layer. Fluentd allows us to collect and consume data for better usage. It decouples the data sources and the backend system by providing a uniform logging layer. It supports more than 500 plugins to connect to various data sources and output streams. Let's understand the configuration of the fluent logging handler:
Configuration File
<source>
type forward
bind localhost
port 24224
</source>
To send logs through fluent using the logstash format, add the following in the salt master configuration file.
fluent_handler:
host: localhost
port: 24224
To send logs through fluent using the Graylog raw JSON format, add the following in the salt master configuration file.
fluent_handler:
host: localhost
port: 24224
payload_type: graylog
tags:
- salt_master.SALT
In the above configuration, the tag option allows us to set descriptive tags for the record being sent. If it is not set, then by default, it is set to "salt". In the case of Graylog, the tag is set to "salt_master", and the "facility" option is set to "SALT".
Log Level: We have the option to set the log level. By default, it is set to the global log level.
Log4mongo Logging Handler
This handler provides a mechanism to send logs to MongoDB. It uses the pymongo driver and is a part of the log4mongo.org project.
To configure the log4mongo handler, put the following code in the minion or master configuration file:
log4mongo_handler:
host: mongodb_host
port: 27017
database_name: ninja-records
collection: ninja_salt_logs
username: coder
password: hey_there
write_concern: 0
log_level: error
Log Level: By default, it takes the value of the global logging level.
Logstash Logging Handler
Logstash is a free and open server-side data processing pipeline that collects data from multiple sources, processes it, and then sends it to our desired "stash". What is a stash? It means keeping your data safe in a secret location. Logstash is dynamic and can handle complex data regardless of the format. It can ingest data of all shapes, sizes and sources.
We have two types of logstash logging handlers.
🌳 UDP Logging Handler
To configure logstash before version 1.2.0 put the following in the salt configuration file.
logstash_udp_handler:
host: 127.0.0.1
port: 9999
version: 0
msg_type: logstash
In the Logstash configuration file, make the following changes:
input {
udp {
type => "udp-type"
format => "json_event"
}
}
For versions after 1.2.0, use the following configurations.
In the salt configuration file:
logstash_udp_handler:
host: 127.0.0.1
port: 9999
version: 1
msg_type: logstash
In the Logstash configuration file:
input {
udp {
port => 9999
codec => json
}
}
🌳 ZeroMQ Logging Handler
ZeroMQ is an asynchronous messaging library specially designed for distributed and concurrent applications. It can run without middleware which makes it handy.
To configure logstash before version 1.2.0 put the following in the salt configuration file.
logstash_zmq_handler:
address: tcp://127.0.0.1:2021
version: 0
In the Logstash configuration file, make the following changes:
input {
zeromq {
type => "zeromq-type"
mode => "server"
topology => "pubsub"
address => "tcp://0.0.0.0:2021"
charset => "UTF-8"
format => "json_event"
}
}
For versions after 1.2.0, use the following configurations.
In the salt configuration file:
logstash_zmq_handler:
address: tcp://127.0.0.1:2021
version: 1
In the Logstash configuration file:
input {
zeromq {
topology => "pubsub"
address => "tcp://0.0.0.0:2021"
codec => json
}
}
Note: The "format" is a critical setting that must be set as a json_event.
Log Level: Both the logstash_udp_handler and logstash_zmq_handler have the option to set the log level. By default, they assume the global log level.
Sentry Logging Handler
Sentry is an open-source error-tracking system that provides deep insights into production bugs. All the stack traces and context variables details are readily available and searchable from the online website. We must have a python library named "Raven" installed in our system to use this library and define the DSN.
Use the following code to configure the DSN under the sentry_handler configuration key:
sentry_handler
dsn: https://pub-key:secret-key@app.getsentry.com/app-id
We can even have complex configuration keys, as shown in the example below.
sentry_handler:
servers:
- https://sentry.ninja.com
- http://192.168.0.1
project: app-id
public_key: sldlvfxmvxlvnxonbofbnxfxnbxllfxk
secret_key: slklvnxflvlvnpsojbvdfnvlxflxnxld
context:
- os
- master
- saltversion
- cpuarch
- ec2.tags.environment
Log Level: The default log level is ERROR for the sentry handler.
Frequently Asked Questions
Why do we need external logging handlers?
The external logging handlers make it easy to use our application's open-source and freely available logging solutions. Salt tightly integrates these handlers and provides an easy interface for our projects.
Why are all the settings done in the salt master?
The salt-master has a configuration file that it can use to give directions to the minions, which do not have any configuration files. The minions can only work according to the order from the master. The pillar files help to give directions to the minions.
What are log levels?
Log levels are a way to define the severity of the error, some errors are mere warnings, and others are critical for the regular operations of the application. These levels tell the log handler what to log and what to skip.
Conclusion
We thoroughly discussed the external logging handlers in salt and saw the different types of external handlers and their configuration and log levels. We saw a sample configuration file for each type of handler. Some brief descriptions of the handlers were also given.
We hope this blog has helped you. We recommend you visit our articles on different SALT topics, such as
🔥 Configuration of SALT Proxy Minions.
🔥 Job Cache in SALT.
🔥 About Delta Proxy Minion.
If you liked our article, do upvote our article and help other ninjas grow. You can refer to our Guided Path on Coding Ninjas Studio to upskill yourself in Data Structures and Algorithms, Competitive Programming, System Design, and many more!
Head over to our practice platform Coding Ninjas Studio to practice top problems, attempt mock tests, read interview experiences and interview bundles, follow guided paths for placement preparations, and much more!!
Happy Reading!!
34+ registered 
