Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
Authorize the Monitoring agent
2.1.
Adding Credentials
2.2.
Creating a service account
2.3.
Copying the private key to your instance
3.
Configure the Monitoring agent
3.1.
Changing the default Monitoring agent configuration
3.2.
Configuring reported CPU metrics
4.
Troubleshoot the Monitoring agent
4.1.
Verifying project and credentials
4.2.
Verifying the agent data
4.3.
Verifying Compute Engine credentials
4.4.
Verifying private-key credentials
5.
Frequently Asked Questions
5.1.
What are your views on Projects?
5.2.
What clearly is Google BigQuery?
5.3.
What is a Google Cloud instance?
6.
Conclusion
Last Updated: Mar 27, 2024

Managing the Monitoring Agent

Author Shivani Singh
0 upvote
Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

Cloud Monitoring accumulates metrics, events, and metadata from Google Cloud, Amazon Web Services (AWS), uptime probes hosted on the cloud, and application instrumentation. You can also collect this data using the BindPlane service from over 150 widely used software components, on-premise systems, and hybrid cloud systems. The Google Cloud operations suite consumes that data and generates insights in the form of dashboards, charts, and alerts. BindPlane is included at no extra cost with your Google Cloud project.

Create an Agent Policy that implements and maintains Google Cloud's operations suite agents across your fleet of VMs to collect metrics data from your Compute Engine instances.

Here in this blog, we will see different approaches to managing the monitoring agent in google cloud. 

managing the monitoring agent

Authorize the Monitoring agent

In this section of the blog, we will see how to set up private-key service account credentials on a VM instance to authorize the Monitoring agent There are also some prerequisites. The agent must be authorized to send data to Monitoring.

Adding Credentials

The method of evaluating a verified client's authorization for a set of resources is referred to as authorization.

The following steps are required to authorize the Monitoring agent on a VM instance:

Step 1: Create a service account with the necessary privileges and private-key credentials in the Google Cloud project associated with your VM instance.

Step 2: This is done in the AWS Extension project that is created when you connect your AWS account for Amazon EC2 VM instances.

Step 3: Copying the private-key credentials to your virtual machine instance, where they will be used as Application Default Credentials for software running on your virtual machine.

Step 4: The agent is being installed or restarted.

Creating a service account

Service accounts can be used for authentication regardless of whether your code runs on Amazon Ec2, App Engine, or on-premise.

Wrap up the Making of a service account processes with the following information to create a service account:

Step 1: Choose the Google Cloud project for which you want to create the service account.

Step 2: Choose the project in which you created the Compute Engine instance.

Step 3: Choose the AWS connector project created when you connected your AWS account to Google Cloud services for Amazon EC2 instances.

Step 4: Choose those roles from the Role drop-down menu:

Step 5: Monitoring Metric Writer > Monitoring This gives the Monitoring agent permission.

If you intend to install the Logging agent, assign it the following role: Logs Writer > Logging This grants permission to the Logging agent.

Step 6: When designing the key, select JSON key type.

Copying the private key to your instance

You must replicate the private-key file to one of the following locations on your VM instance after creating the service account so that the agent can recognize the credentials. You can use any file-copying program you want.

  • Linux only: /etc/google/auth/application_default_credentials.json
  • Windows only: C:\ProgramData\Google\Auth\application_default_credentials.json
  • For both Linux and Windows: Any location you store in the variable, GOOGLE_APPLICATION_CREDENTIALS. The variable must be visible to the agent's process.
Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Configure the Monitoring agent

In this section of the blog, we will see how to configure which assembled metrics are collected by the Monitoring agent from a Linux VM

When the Monitoring agent is assembled on a Windows VM, it cannot be configured.

Changing the default Monitoring agent configuration

To modify the Monitoring agent's configuration, do the following:

Edit the monitoring configuration file using the command: /etc/stackdriver/collectd.conf.

Then Restart the Monitoring agent by running the following command: sudo service stackdriver-agent restart

Configuring reported CPU metrics

By default, the Monitoring agent uses the agent.googleapis.com/cpu/utilization metric type to report total CPU utilization by the state for all CPUs on the VM. You must change these settings if you want the advisor to report CPU utilization per CPU or CPU usage time.

Do the following to customize the Monitoring agent to notify the agent.googleapis.com/cpu/utilization per CPU:

  • Leave ValuesPercentage set to true by default.
  • Turn on ReportByCpu.

Do the following to configure the Monitoring agent to report the agent.googleapis.com/cpu/usage time per CPU:

  • ValuesPercentage should be set to false.
  • Turn on ReportByCpu.

The Monitoring agent can report either CPU utilization or CPU usage, but not both. This is a known collected limitation. 

Troubleshoot the Monitoring agent

In this section of the blog, we will discuss problems in the installation or running of the Monitoring agent. 

Verifying project and credentials

If the agent reports access or authorization errors, or if the agent appears to be running normally but there is no data or your alerting policies aren't working as expected, you should double-check your VM instance's credentials, including whether they specify the correct project:

  • Try reading some of the time series data to see if data is arriving in Monitoring.
  • If you're using a Compute Engine VM instance with standard (rather than private-key) credentials, it's unlikely that data is being routed to the wrong project, but your credentials could still be inadequate.
  • If you're using an Amazon EC2 VM instance or private-key login information on your Amazon Ec2 instance, the credentials may be invalid or from the wrong project. For AWS accounts, the project used by the agent has to be the AWS connector project.

If the problem persists, reinstall the monitoring agent.

Verifying the agent data

To ensure that the agent sends metrics correctly, use the Monitoring API's timeSeries.list method to look for the latest time series data from the VM instance.

Here are the specifics for using the timeSeries.list method:

Step 1: Determine the instance ID of the VM instance where the agent was installed:

  • Instances of the Compute Engine: Navigate to the Compute Engine instance detail page. Click Equivalent REST at the bottom of the page. The ID consists of a 19-digit number.
  • Amazon EC2 instances: Each instance's ID is displayed in the list of instances. The ID appears to be i-1a2b3c4d.
     

Step 2: Go to the timeSeries.list method's documentation page.

Step 3: Fill out the APIs Explorer form as follows: 

  • Prefix the name with projects/ to the project containing your VM instance. You need to use the AWS connector project for your Account online to use Amazon EC2 instances.
  • To select an agent metric from your VM instance, add a filter to the following line. Copy it and paste it into the APIs Explorer, then modify the VM instance ID: metric. type = "agent.googleapis.com/memory/bytes_used" AND resource. label.instance_id = "[YOUR-VM-INSTANCE-ID]"
  • Set the time interval for the search. You want about a five-minute interval:
  • Set the interval.endTime property to the current GMT time, which can be found at time.is/GMT. The time must be formatted as in the example below. Don't use quotation marks around the time: 2016-10-31T14:10:00Z
  • Using the same format, set the interval.startTime to about five minutes before the end time.
  • All other fields should be left blank.

Step 4: Click the Execute button.

The output should look something like this:

{
 "timeSeries": [
  {
   "metric": {
    "labels": {
     "state": "buffered"
    },
    "type": "agent.googleapis.com/memory/bytes_used"
   },
   "resource": {
    "type": "[GCP-OR-AWS-INSTANCE-TYPE]",
    "labels": {
     "instance_id": "[YOUR-VM-INSTANCE-ID]",
     "zone": "[YOUR-INSTANCE-ZONE]",
     "project_id": "[YOUR-PROJECT-ID]"
    }
   },
   "metricKind": "GAUGE",
   "valueType": "DOUBLE",
   "points": [
    {
     "interval": {
      "startTime": "[START_TIME]",
      "endTime": "[END_TIME]"
     },
     "value": {
      "doubleValue": 27451392
     }
    },
    ...

If the API call returns any time series from your VM instance, as shown above, your agent is operational and you are done.

If you don't see any time series, look here:

  • If your API call returns an error message, this does not indicate a problem with the agent. Check that the APIs Explorer fields are correctly filled out:
  • "Invalid argument" errors most likely indicate a spelling or formatting issue with the project ID, filter, or the two timestamps.
  • The time-stamp arguments' requirements vary depending on the metric type. Metric data can be GAUGE, DELTA, or CUMULATIVE.
  • Both the start and end times are required for DELTA and CUMULATIVE metrics, and the end time must be later than the start time. Because these metric types record changes over time, the start and end times must be specified at non-zero intervals.
  • "Not authorized" errors can occur when the project ID is misspelled.
  • "Not found" errors can occur when you fail to include the required projects/prefix in the "name" field.
  • Resolve the issues and try the API call afterward.

Double-check that your filter and intervals are correct if the API call is successful but you only receive an empty response.

Verifying Compute Engine credentials

Use the Google Cloud console's Compute Engine VM instances page to ensure that your Compute Engine VM instance has sufficient credentials for the Monitoring agent.

  • Change the current Google Cloud project to the one associated with your Compute Engine VM instance if needed. 
  • Click the name of your VM instance on the VM Instances page. The VM instance's detail page appears.
  • Look under the Cloud API access scopes heading on the VM instance details page:
  • If you see "Allow full access to all Cloud APIs," your credentials are sufficient.
  • You have sufficient credentials if you see Write Only or Full permission next to Stackdriver Monitoring API, an older name for the Cloud Monitoring API.
  • Otherwise, your instance's default service account lacks the necessary credentials for the agent. You must add private-key service account credentials to your instance before you can use the agent.

Verifying private-key credentials

To ensure that valid private-key credentials are installed on your virtual machine, first ensure that the credentials file exists in the expected location, and then ensure that the information in the credentials file is correct. Previously valid credentials can be revoked via the Google Cloud console's IAM & Admin > Service accounts section.

To see if the credentials are present in your instance, run the following command: sudo cat $GOOGLE_APPLICATION_CREDENTIALS

sudo cat /etc/google/auth/application_default_credentials.json

If it displays some file, then your credentials are valid. 

To know if the credentials are credible or not: In the credentials file, project id refers to your Google Cloud project, client email refers to the project's service account, and private key id refers to the service account's private key. Compare this information to what is displayed in the Google Cloud console's IAM & Admin > Service accounts section.

If the credentials are invalid, do the following to generate new credentials.

  • Go to Monitoring Projects to see a list of projects associated with your metrics scope.
  • Select Settings from the Monitoring navigation pane, then the Summary tab:
  • For AWS, follow the link to the Google Cloud console for the project in question.
  • Navigate to the Google Cloud console after identifying the project containing the Compute Engine resources in question.
  • Navigate to the IAM Service Accounts page in the Google Cloud console.
  • Create a new service account and corresponding private key.
  • Replace the private key on the instances associated with the service account.
  • Relaunch the agent.

Frequently Asked Questions

What are your views on Projects?

The projects are the containers for Google Compute Engine resources. There are some key points about projects which include The projects are divided into compartments. The projects are not intended for resource sharing. They may have multiple users and owners. Each project's billing is handled separately. The projects are no longer accessible to one another.

What clearly is Google BigQuery?

Google BigQuery is a hardware replacement for traditional data warehouses. It is used as a data warehouse, acting as a central repository for all analytical data in an organization. Furthermore, BigQuery divides the data table into units known as datasets.

What is a Google Cloud instance?

A Google Cloud console project may contain one or more instances, each of which is associated with one or more projects. Instances of a project can be created on several different computer systems and machine types.

Conclusion

To sum it up, in this blog, we discussed managing the monitoring agent. We also discussed various operations like authorizing the agent, configuring the monitoring agent, and last but not least, troubleshooting the monitoring agent. 

For more content, Refer to our guided paths on Coding Ninjas Studio to upskill yourself.

Do upvote our blogs if you find them helpful and engaging!

Happy Learning!

Previous article
Overview of Monitoring Agent
Next article
Overview of uptime checks
Live masterclass