Managing the Monitoring Agent

Configure the Monitoring agent

In this section of the blog, we will see how to configure which assembled metrics are collected by the Monitoring agent from a Linux VM

When the Monitoring agent is assembled on a Windows VM, it cannot be configured.

Changing the default Monitoring agent configuration

To modify the Monitoring agent's configuration, do the following:

Edit the monitoring configuration file using the command: /etc/stackdriver/collectd.conf.

Then Restart the Monitoring agent by running the following command: sudo service stackdriver-agent restart

Configuring reported CPU metrics

By default, the Monitoring agent uses the agent.googleapis.com/cpu/utilization metric type to report total CPU utilization by the state for all CPUs on the VM. You must change these settings if you want the advisor to report CPU utilization per CPU or CPU usage time.

Do the following to customize the Monitoring agent to notify the agent.googleapis.com/cpu/utilization per CPU:

• Leave ValuesPercentage set to true by default.
• Turn on ReportByCpu.

Do the following to configure the Monitoring agent to report the agent.googleapis.com/cpu/usage time per CPU:

• ValuesPercentage should be set to false.
• Turn on ReportByCpu.

The Monitoring agent can report either CPU utilization or CPU usage, but not both. This is a known collected limitation. 

Troubleshoot the Monitoring agent

In this section of the blog, we will discuss problems in the installation or running of the Monitoring agent. 

Verifying project and credentials

If the agent reports access or authorization errors, or if the agent appears to be running normally but there is no data or your alerting policies aren't working as expected, you should double-check your VM instance's credentials, including whether they specify the correct project:

• Try reading some of the time series data to see if data is arriving in Monitoring.
• If you're using a Compute Engine VM instance with standard (rather than private-key) credentials, it's unlikely that data is being routed to the wrong project, but your credentials could still be inadequate.
• If you're using an Amazon EC2 VM instance or private-key login information on your Amazon Ec2 instance, the credentials may be invalid or from the wrong project. For AWS accounts, the project used by the agent has to be the AWS connector project.

If the problem persists, reinstall the monitoring agent.

Verifying the agent data

To ensure that the agent sends metrics correctly, use the Monitoring API's timeSeries.list method to look for the latest time series data from the VM instance.

Here are the specifics for using the timeSeries.list method:

Step 1: Determine the instance ID of the VM instance where the agent was installed:

• Instances of the Compute Engine: Navigate to the Compute Engine instance detail page. Click Equivalent REST at the bottom of the page. The ID consists of a 19-digit number.
• Amazon EC2 instances: Each instance's ID is displayed in the list of instances. The ID appears to be i-1a2b3c4d.
   

Step 2: Go to the timeSeries.list method's documentation page.

Step 3: Fill out the APIs Explorer form as follows: 

• Prefix the name with projects/ to the project containing your VM instance. You need to use the AWS connector project for your Account online to use Amazon EC2 instances.
• To select an agent metric from your VM instance, add a filter to the following line. Copy it and paste it into the APIs Explorer, then modify the VM instance ID: metric. type = "agent.googleapis.com/memory/bytes_used" AND resource. label.instance_id = "[YOUR-VM-INSTANCE-ID]"
• Set the time interval for the search. You want about a five-minute interval:
• Set the interval.endTime property to the current GMT time, which can be found at time.is/GMT. The time must be formatted as in the example below. Don't use quotation marks around the time: 2016-10-31T14:10:00Z
• Using the same format, set the interval.startTime to about five minutes before the end time.
• All other fields should be left blank.

Step 4: Click the Execute button.

The output should look something like this:

{
 "timeSeries": [
  {
   "metric": {
    "labels": {
     "state": "buffered"
    },
    "type": "agent.googleapis.com/memory/bytes_used"
   },
   "resource": {
    "type": "[GCP-OR-AWS-INSTANCE-TYPE]",
    "labels": {
     "instance_id": "[YOUR-VM-INSTANCE-ID]",
     "zone": "[YOUR-INSTANCE-ZONE]",
     "project_id": "[YOUR-PROJECT-ID]"
    }
   },
   "metricKind": "GAUGE",
   "valueType": "DOUBLE",
   "points": [
    {
     "interval": {
      "startTime": "[START_TIME]",
      "endTime": "[END_TIME]"
     },
     "value": {
      "doubleValue": 27451392
     }
    },
    ...

If the API call returns any time series from your VM instance, as shown above, your agent is operational and you are done.

If you don't see any time series, look here:

• If your API call returns an error message, this does not indicate a problem with the agent. Check that the APIs Explorer fields are correctly filled out:
• "Invalid argument" errors most likely indicate a spelling or formatting issue with the project ID, filter, or the two timestamps.
• The time-stamp arguments' requirements vary depending on the metric type. Metric data can be GAUGE, DELTA, or CUMULATIVE.
• Both the start and end times are required for DELTA and CUMULATIVE metrics, and the end time must be later than the start time. Because these metric types record changes over time, the start and end times must be specified at non-zero intervals.
• "Not authorized" errors can occur when the project ID is misspelled.
• "Not found" errors can occur when you fail to include the required projects/prefix in the "name" field.
• Resolve the issues and try the API call afterward.

Double-check that your filter and intervals are correct if the API call is successful but you only receive an empty response.

Verifying Compute Engine credentials

Use the Google Cloud console's Compute Engine VM instances page to ensure that your Compute Engine VM instance has sufficient credentials for the Monitoring agent.

• Change the current Google Cloud project to the one associated with your Compute Engine VM instance if needed. 
• Click the name of your VM instance on the VM Instances page. The VM instance's detail page appears.
• Look under the Cloud API access scopes heading on the VM instance details page:
• If you see "Allow full access to all Cloud APIs," your credentials are sufficient.
• You have sufficient credentials if you see Write Only or Full permission next to Stackdriver Monitoring API, an older name for the Cloud Monitoring API.
• Otherwise, your instance's default service account lacks the necessary credentials for the agent. You must add private-key service account credentials to your instance before you can use the agent.

Verifying private-key credentials

To ensure that valid private-key credentials are installed on your virtual machine, first ensure that the credentials file exists in the expected location, and then ensure that the information in the credentials file is correct. Previously valid credentials can be revoked via the Google Cloud console's IAM & Admin > Service accounts section.

To see if the credentials are present in your instance, run the following command: sudo cat $GOOGLE_APPLICATION_CREDENTIALS

sudo cat /etc/google/auth/application_default_credentials.json

If it displays some file, then your credentials are valid. 

To know if the credentials are credible or not: In the credentials file, project id refers to your Google Cloud project, client email refers to the project's service account, and private key id refers to the service account's private key. Compare this information to what is displayed in the Google Cloud console's IAM & Admin > Service accounts section.

If the credentials are invalid, do the following to generate new credentials.

• Go to Monitoring Projects to see a list of projects associated with your metrics scope.
• Select Settings from the Monitoring navigation pane, then the Summary tab:
• For AWS, follow the link to the Google Cloud console for the project in question.
• Navigate to the Google Cloud console after identifying the project containing the Compute Engine resources in question.
• Navigate to the IAM Service Accounts page in the Google Cloud console.
• Create a new service account and corresponding private key.
• Replace the private key on the instances associated with the service account.
• Relaunch the agent.

Frequently Asked Questions

What are your views on Projects?

The projects are the containers for Google Compute Engine resources. There are some key points about projects which include The projects are divided into compartments. The projects are not intended for resource sharing. They may have multiple users and owners. Each project's billing is handled separately. The projects are no longer accessible to one another.

What clearly is Google BigQuery?

Google BigQuery is a hardware replacement for traditional data warehouses. It is used as a data warehouse, acting as a central repository for all analytical data in an organization. Furthermore, BigQuery divides the data table into units known as datasets.

What is a Google Cloud instance?

A Google Cloud console project may contain one or more instances, each of which is associated with one or more projects. Instances of a project can be created on several different computer systems and machine types.

Conclusion

To sum it up, in this blog, we discussed managing the monitoring agent. We also discussed various operations like authorizing the agent, configuring the monitoring agent, and last but not least, troubleshooting the monitoring agent. 

For more content, Refer to our guided paths on Coding Ninjas Studio to upskill yourself.

Do upvote our blogs if you find them helpful and engaging!

Happy Learning!