Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
Logging System in Linux
2.1.
Services provided by the logging system
2.2.
Types of Logs in Linux
3.
Some Crucial Troubleshooting
3.1.
Login Failures
3.2.
Memory Problems
3.3.
System Crash
3.4.
Shutdown Concerns
4.
Frequently Asked Questions
4.1.
What is a logging system?
4.2.
What are the different types of logging systems?
4.3.
What services are provided by the logging system in Linux?
4.4.
How troubleshooting with Linux logs can be used for login failures?
4.5.
How memory problems can be resolved using Linux logs?
5.
Conclusion
Last Updated: Mar 27, 2024
Easy

Troubleshooting with Linux Logs

Create a resume that lands you SDE interviews at MAANG
Speaker
Anubhav Sinha
SDE-2 @
12 Jun, 2024 @ 01:30 PM

Introduction

Troubleshooting is one of the most important and crucial tasks for the administrator of Linux, as the issues can be created anytime, and the administrator needs to diagnose it to understand the cause. Log Files can be used at this point that contain messages about the Linux system, including the kernel, services, and applications running on the system. So directly starting with the troubleshooting, we will do a quick introduction to what a logging system is.

Troubleshooting with Linux Logs

In the article “Troubleshooting with Linux Logs”, we will understand what is the logging system in Linux, the services provided by system logs, types of logs in Linux. Then, we will discuss some crucial troubleshooting with Linux logs, such as login failures, memory problems, system crashes (or reboots), and shutdown concerns.

Read about Linux System Logging and Troubleshooting.

Logging System in Linux

In the article Troubleshooting with Linux Logs, we will discuss what a logging system is. So Logging is nothing but a facility provided by Linux by which users can track the information related to their system. There are different log files that basically contain the information which helps in troubleshooting issues. The logging system is based on a built-in protocol called syslog. The syslog is a standardized way of creating and sending the logs from the operating system to the syslog server.

The log files can be found in the directory /var/log. The syslog directory is also present in the log directory.

Services provided by the logging system

There are several services provided by the logging system in Linux, which are as follows:

  1. Diagnosing Performance Issues
    Performance issues like memory leaks and disk input output bottlenecks can be diagnosed by the logging system in Linux.
     
  2. Monitoring System Health
    Logging systems in Linux can also be used to monitor the system's health and also detect issues before they become critical.
     
  3. Compliance and Auditing
    The logging system also maintains the logs for compliance and auditing for many organizations by providing a record of system activity.
     
  4. Security
    Logging systems can also be used for the security issues such as unauthorized access attempts or any suspicious activity. So the administrator can identify and respond to these security issues as early as possible.

Types of Logs in Linux

In this section, we will discuss some important logs in Linux. Here are some of the following:

  1. System Logs
    This type of log contains information about the system’s operation, such as kernel messages, boot messages, and hardware events.
     
  2. Application Logs
    These logs contain information about the behavior of an application, including errors.
     
  3. Service Logs
    These logs contain information about all the services running on the system.
     
  4. Event Logs
    These logs contain information about the events, such as user logins, security events, etc.
Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Some Crucial Troubleshooting

In this section of the article Troubleshooting with Linux Logs, we will discuss some of the crucial troubleshooting examples using Linux logs. Here are the following:

Login Failures

If there are some users that have logged in or attempted to log in to your system, authentication logs can be checked that show the failure attempts. These authentication events are logged by the pluggable authentication module (PAM); if the authentication is successful, then the events will contain the strings such as “Accepted Password” and “Session Opened”. If the authentication gets failed, then the strings contained by the authentication events will be “Failed Password” and “User Unknown”.

To check the authentication failure message, the below command can be used:

sudo grep "authentication failure" /var/log/auth.log

 

For example, if the authentication is failed, then the failure events will look something like below:

pam_unixx(sshd:auth): authentication failure; logname= uid=2 euid=0 tty=ssh ruser= rhost=8.3.2.2
Failed password for invalid user NinjaXYZ from 10.0.2.2 port 5000 ssh4
pam_unix(sshd:auth): check pass; user unknown
PAM service(sshd) ignoringg max retrie; 4 > 3

 

Else If the authentication is successful, then successful events will look something like below:

Accepted password for NinjaXYZ from 8.3.2.2 port 5001 ssh4
pam_unix(sshd:session): session opened for user NinjaXYZ by (uid=2)
pam_unix(sshd:session): session closed for user NinjaXYZ

Memory Problems

A server might crash if there is a memory problem in the system, and the main problem can occur in memory when the RAM and swap space are completely exhausted. The kernel starts killing processes that are using the most memory. When the system uses, all the memory and a process tries to access the additional memory. Now what you can troubleshoot here is which process is killed instead of allowing that process to crash.

To check the process with the string “Out of memory” can be used as given in the below command:

sudo grep "Out of memory" /var/log/syslog

 

Here is an example output for the above showing that “process 16471” is being killed:

July 21 21:30:26 ip-172-31-34-37 kernel: [ 1575.404070] Out of memory: Kill process 16471 (NinjaXYZ) score 838 or sacrifice child
July 21 21:30:26 ip-172-31-34-37 kernel: [ 1575.408946] Killed process 16471 (NinjaXYZ) total-vm:144200240kB, anon-rss:562316kB, file-rss:0kB, shmem-rss:0kB
July 21 21:30:27 ip-172-31-34-37 kernel: [ 1575.518686] oom_reaper: reaped process 16471 (NinjaXYZ), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

System Crash

The problem can occur due to the system crash or reboot, and the method which can be used to display the logins for the “reboot” pseudo user. There is the command “last reboot”, which can be used here, as given below:

sudo last reboot


Here is an example output for the above showing the previous system reboots:

reboot system boot 5.4.0-80-generic Fri Aug 10 15:25 still running
reboot system boot 5.4.0-80-generic Thu Aug 12 09:10 still running
reboot system boot 5.4.0-80-generic Wed Aug 17 17:30 still running
reboot system boot 5.4.0-80-generic Tue Aug 19 14:55 still running
reboot system boot 5.4.0-80-generic Mon Aug 24 08:20 still running

Shutdown Concerns

If there are some problems and the administrator wants to see the log files, if someone ran the shutdown command manually. The below command can be used to check:

last | grep "ubuntu.*shutdown"

 

Here is an example output for the above showing if someone has used the shutdown command manually.

ubuntu   pts/0      203.0.113.55    Sat Aug 21 14:30 - 14:35  (00:05)     shutdown system down
ubuntu   pts/0      203.0.113.55    Fri Aug 24 09:15 - 09:20  (00:05)     shutdown system down
ubuntu   pts/1     198.51.100.20   Thu Aug 29 16:40 - 16:45  (00:05)     shutdown system down

 

Read about Linux System Logging and Troubleshooting.

Frequently Asked Questions

What is a logging system?

Logging is nothing but a facility provided by Linux by which users can track the information related to their system. There are different log files that basically contain the information which helps in troubleshooting issues.

What are the different types of logging systems?

There are different types of logs in Linux, such as System Logs, which contain information about the system’s operation. Application Logs contain information about the behavior of an application, and Service Logs contain information about all the services running on the system.

What services are provided by the logging system in Linux?

There are several services, such as the performance issues like memory leaks and disk input output bottlenecks that can be diagnosed, the system's health and also detection issues before they become critical, the logging system maintains the logs for compliance and auditing for many organizations.

How troubleshooting with Linux logs can be used for login failures?

If there are some users that have logged in or attempted to log in, authentication logs can be checked that show the failure attempts. To check the authentication failure message, the command can be used: sudo grep "authentication failure" /var/log/auth.log.

How memory problems can be resolved using Linux logs?

A problem can occur in memory when the RAM and swap space are completely exhausted. The kernel starts killing processes that are using the most memory. To check these processes, the following command can be used: sudo grep "Out of memory" /var/log/syslog.

Conclusion

There are different log files that basically contain the information which helps in troubleshooting issues. Logging is nothing but a facility provided by Linux by which users can track the information related to their system. In the article “Troubleshooting with Linux Logs”, we first discussed what is the logging system in Linux along with its services and its types. We also discussed some crucial troubleshooting that can be done using Linux logs.

Here are more articles that are recommended to read:

You can refer to our guided paths on the Coding Ninjas. You can check our course to learn more about DSADBMSCompetitive ProgrammingPythonJavaJavaScript, etc. 

Also, check out some of the Guided Paths on topics such as Data Structure and AlgorithmsCompetitive ProgrammingOperating SystemsComputer Networks, DBMS, and System Design, etc. as well as some Contests, Test SeriesInterview Bundles, and some Interview Experiences curated by top Industry Experts.

Happy Learning!

Previous article
Tail Command in Linux
Next article
Tar Command in Linux with Examples
Live masterclass