Advanced concepts of metrics in cloud monitoring

AWS connector projects

A Google Cloud project called an AWS connection project enables Cloud Monitoring to read metrics for a particular AWS account. A Google Cloud project with an AWS connection project as a monitored project is shown in the diagram below. The project's AWS connection reads metrics from an AWS account and stores them:

 

Setup before you begin

• Having an AWS account is required.
• To make your AWS connection projects identifiable, decide on a naming scheme. We advise including identifying information about the AWS account it monitors in the project name of an AWS connector. The AWS account that an AWS connection project is watching cannot be changed.
• Check to see if Google Cloud is connected to your AWS account.
• Make sure you have all the necessary permissions to build a Google Cloud project and that Identity and Access Management (IAM) role on scoping project allows you to adjust its metrics scope:

Connecting your AWS account to Google Cloud

1. Select Monitoring from the Google Cloud console or press the following button:

Go to Monitoring 

2. To examine or change the metrics scope of a Cloud project, use the project picker in the Google Cloud console.

3. Select Settings from the Monitoring menu panel.

4. Click the Create AWS connector project button on the Settings page.

 

 

5. Select a project by clicking on the Create a connector project step.

6. Choose New Project from the dialog's options, then finish the new project dialogue.

7. To proceed to the Authorize AWS for Monitoring step, click Next.

8. Making an Amazon IAM role: 

•  Log into your AWS account in a new window, choose the IAM page, and click Roles.
• Choose Create Role.
• Choose a different AWS account.
• Enter the account ID found on the Google Cloud console's Authorize AWS for Monitoring page in the Account ID text box.
• Decide on requiring an external ID.
• Enter the external ID found on the Google Cloud console's Authorize AWS for Monitoring page in the External ID text box.
• After clearing Require MFA, select Next: Permissions.
• Enter "ReadOnlyAccess" and click "ReadOnlyAccess" in the permissions search box.
• Make sure Create role without a permissions boundary is selected in the expanded Set Permission Boundary section.
• Click Next: Tags.
• Click Next: Review.
• Click Create Role after providing the role's name and description.
• To access the Summary page of the role you established, select it. The Role ARN should be copied to your clipboard.
• Paste the AWS Role ARN into the Role ARN text box in the Google Cloud console, then click Add AWS Account.    

Viewing AWS metrics in multiple metrics scopes

Do the following to include an existing AWS connector project in a metrics scope:

1. Select Monitoring from the Google Cloud console or press the following button:

Go to Monitoring 

2. Do the following for each metric scope you want to utilise to observe your AWS metrics:

• To choose the scoping project for the metrics scope, use the project picker in the Google Cloud console.
• Select Settings from the Monitoring menu panel.
• Click on Add AWS connector project in the AWS Accounts in the scope window.
• Click Add projects after selecting the AWS connector projects you want to include.

Removing AWS connector projects from a metrics scope

Do the following to remove AWS connector projects:

1. Select Monitoring from the Google Cloud console or press the following button:

Go to Monitoring 

2.  To examine or change the metrics scope of a Cloud project, use the project picker in the Google Cloud console.

3. Select Settings from the Monitoring menu panel.

4. Choose the AWS connector projects you want to remove from the AWS Accounts in the scope window, and then click Remove project.

5. Delete any AWS connection projects that you've deleted and aren't currently being tracked.

List of Monitored projects

Do the following to show a list of monitored projects:

1. Select Monitoring from the Google Cloud console or press the following button:

Go to Monitoring 

2. To examine or change the metrics scope of a Cloud project, use the project picker in the Google Cloud console.

3. Click on >Expand on the Metrics scope field of Monitoring navigation pane.

The page that appears when the AllEnvironments project is selected is depicted in the next screenshot:

Additionally, it demonstrates that this project consists of two monitored projects, Staging, and Production.

Selecting a different metrics scope

Use the project picker in the Google Cloud console to choose a different project if you want to choose a different metrics scope.

Now let's have a look at View metrics for AWS accounts

Viewing metrics for AWS accounts

In this, we are going to learn how to use Cloud Monitoring to view and track metrics from Amazon EC2.

Stopping ingestion of AWS account metrics

Delete the AWS connector project for the account to halt the ingestion of metrics and logs from that AWS account.

Manage metrics scopes by using the API

In this, we are going to learn about how to manage the metrics scope of a Google Cloud project using the metrics scope methods in the Cloud Monitoring API.

curl command parameters

The metrics scopes APIs are immediately accessible. On this page, you may find examples of curl commands. The URL of an API resource is followed by a set of arguments in each curl command:

curl -H "Authorization: Bearer ${TOKEN}" <other_args> \
https://monitoring.googleapis.com/v1/locations/global/metricsScopes/<resource>

These environment variables are used in the examples on this page:

• TOKEN: Stores the authentication token.
• SCOPING_PROJECT_ID_OR_NUMBER: Conserves the project ID or number for a metrics scope scoping project.
• MONITORED_PROJECT_ID_OR_NUMBER: This contains a project's ID or number intended to be added to or purged from a metrics scope.

Get a metrics scope

To retrieve information about metric scopes.Send the GET request to the locations.global.metricsScopes.listMetricsScopesByMonitoredProject

curl -H "Authorization: Bearer ${TOKEN}" \
https://monitoring.googleapis.com/v1/locations/global/metricsScopes/${SCOPING_PROJECT_ID_OR_NUMBER}

Upon success, a MetricsScope object is returned.

All metrics scopes list in a project.

To retrieve list of all metrics scopes.Send the GET request to the locations.global.metricsScopes.listMetricsScopesByMonitoredProject 

curl -H "Authorization: Bearer ${TOKEN}" \
https://monitoring.googleapis.com/v1/locations/global/metricsScopes:listMetricsScopesByMonitoredProject?monitored_resource_container=projects/${PROJECT_ID_OR_NUMBER}

Upon success, a MetricsScope object is returned.

Add a project to a metrics scope.

Send a POST request to, locations.global.metricsScopes.projects.create to add a Google Cloud project to a metrics scope.

curl -H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" -X POST \
-d "{'name': 'locations/global/metricsScopes/${SCOPING_PROJECT_ID_OR_NUMBER}/projects/${MONITORED_PROJECT_ID_OR_NUMBER}'}" \
https://monitoring.googleapis.com/v1/locations/global/metricsScopes/${SCOPING_PROJECT_ID_OR_NUMBER}/projects

This asynchronous method returns an Operation object as its result.

A successful attempt to add a monitored project will result in the response shown in the example below:

{
  "name": "operations/6915efde-1915-400a-ad49-7b62041d9bd2",
  "metadata": {
    "@type": "type.googleapis.com/google.monitoring.metricsscope.v1.OperationMetadata",
    "state": "DONE",
    ...
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.monitoring.metricsscope.v1.MonitoredProject",
    "name": "locations/global/metricsScopes/012012012012/projects/678678678678",
    "provider": "GCP",
    "providerAccountId": "...",
    ...
  }
}

Removing a project from a metrics scope

Send a DELETE request to locations.global.metricsScopes.projects.delete to add a Google Cloud project to a metrics scope.

curl -H "Authorization: Bearer ${TOKEN}" -X DELETE \
https://monitoring.googleapis.com/v1/locations/global/metricsScopes/${SCOPING_PROJECT_ID_OR_NUMBER}/projects/${MONITORED_PROJECT_ID_OR_NUMBER}

This asynchronous method returns an Operation object as its result.

Successful removal of a monitored project will result in the reaction shown in the example below:

{
  "name": "operations/4367ff34-0ff0-4767-b8d3-0638e30f077c",
  "metadata": {
    "@type": "type.googleapis.com/google.monitoring.metricsscope.v1.OperationMetadata",
    "state": "DONE",
    ...
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.protobuf.Empty"
  }
}

Asynchronous API methods

The Cloud Monitoring API's metrics scope methods that alter the system's state, such as the command to add a monitored project to a metrics scope, are all asynchronous. The command response for these commands is an Operation object.

Send a GET request to the operation.get to update the information on the status

curl -H "Authorization: Bearer ${TOKEN}" \
https://monitoring.googleapis.com/v1/${OPERATION_NAME}

curl command setup

To make the generation of the curl commands simpler, set the following environment variables:

•  To store the scoping project ID or number, create the following environment variable:

SCOPING_PROJECT_ID_OR_NUMBER=a-sample-project

• Log in with the Google Cloud CLI:

gcloud auth login

• Establish an authorization token, then store it in an environment variable:

TOKEN=`gcloud auth print-access-token`

• Echo the TOKEN variable to confirm that an access token was received:

echo ${TOKEN}
ya29.GluiBj8o....

Disable the Cloud Monitoring API

In this, you'll get to learn how to make a Google Cloud project's Cloud Monitoring API inactive.

Do the following to disable the Cloud Monitoring API:

Google cloud console

1. Go to the APIs & Services page by first selecting the Cloud project or the AWS connection project in the Google Cloud console:

Go to APIs & Services 

2. Choose  Cloud Monitoring API.

3. Click API Disable.

Gcloud CLI

To disable the Cloud Monitoring API in your current project, enter the following command:

gcloud services disable monitoring.googleapis.com

Frequently Asked Questions

What is called a cloud?

The term "the cloud" describes the software and databases that run on servers that may be accessed via the Internet.

What is the Cloud Controls Matrix?

The Cloud Controls Matrix (CCM) from the Cloud Security Alliance is a tool created expressly to help potential cloud customers and cloud vendors analyze the overall security risk of a cloud provider. 

What is an Asynchronous method?

The execution of an asynchronous method occurs in a separate thread from the main application thread.

What is cloud bursting?

Hybrid clouds are related to cloud bursting. The notion is that a certain application typically runs in a local computer environment or a private cloud.

What is IAM?

Identity and access management (IAM) is a centralized and consistent method to automate access controls, maintain user identities, and adhere to compliance standards in both traditional and containerized settings.

Conclusion

This blog has extensively discussed Advanced concepts of metrics in cloud monitoring, AWS connector projects, Disabled the Cloud Monitoring API, etc. We hope this blog has helped you in enhancing your knowledge about Advanced concepts of metrics in cloud monitoring. If you want to learn more, check out the excellent content on the Coding Ninjas Website: Key concepts of cloud logging

Refer to our guided paths on the Coding Ninjas Studio platform to learn more about DSA, DBMS, Competitive Programming, Python, Java, JavaScript, etc. 

Refer to the links problems, top 100 SQL problems, resources, and mock tests to enhance your knowledge.

For placement preparations, visit interview experiences and interview bundle.

Do upvote our blog to help other ninjas grow. Happy Coding!