Monitor a compute engine virtual machine.

Monitor a Compute Engine virtual machine

You perform the following in this quickstart:

1. A Compute Engine VM instance should be created.
2. Activate the Apache HTTP Server.
3. The Cloud Monitoring and Logging agents should be installed.
4. Make a check for uptime that includes an alerting policy.
5. Make a dashboard and chart that are unique.
6. Observe your logs.
7. Cleaning up

Creating a Compute Engine instance

1.Go to Compute in the Google Cloud console, and then pick Compute Engine:

   Go to Compute Engine

2. Click Create to start a new VM instance.

3. For your instance, complete the fields as follows:

• Enter lamp-1-vm in the Name field.
• Select Small in the Machine type field.
• Make sure Debian GNU/Linux is configured on the boot disk.
• Select Allow HTTP traffic as well as Allow HTTPS traffic in the Firewall area.

4. Press Create. On the VM Instances page, wait a few minutes for your instance to start up.

5. Select SSH under Connect in the Connect column to launch a terminal to your instance.

6. Update your instance's package lists

sudo apt-get update

7. Set up the HTTP server Apache2.

sudo apt-get install apache2 php7.0

8. Open your browser and go to http://[External IP] to connect to your Apache2 HTTP server. The external IP address of your Compute Engine instance should be used in place of [External IP]. You see the default page for Apache2:

Installing agents

The VM instance's logs and metrics are delivered to Cloud Monitoring and Logging by the agents:

1.Either open a new terminal or switch to the one connected to your VM instance.

2. Activate the Cloud Monitoring agent after installation:

• Update the package list and add the package repository:

curl -sSO https://dl.google.com/cloudagents/add-monitoring-agent-repo.sh
sudo bash add-monitoring-agent-repo.sh
sudo apt-get update

• Set up the agent:

sudo apt-get install stackdriver-agent

• Activate the agent:

sudo service stackdriver-agent start

3. The Cloud Logging agent should be set up, configured, and launched:

• Update the package list and add the package repository:

curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh
sudo bash add-logging-agent-repo.sh
sudo apt-get update

• Set up the agent:

sudo apt-get install google-fluentd                          

• Install the default agent configuration for Cloud Logging to ingest structured data:

sudo apt-get install google-fluentd-catch-all-config-structured

• Activate the agent:

sudo service google-fluentd start

Create an uptime check and an alerting policy

Follow these steps to create an uptime check:

1.Select Monitoring from the Google Cloud console menu.

Go to Monitoring

2. Click "Uptime checks" in the navigation pane.

3. Create Uptime Check by clicking.

4. Enter My Uptime Check for the title, then click Next.

5. Target:

• Make HTTP your protocol of choice.
• The resource type should be set to an instance.
• Choose Single in the Applies To field and lamp-1-vm as the instance name.
• Click Next, leaving every other field's default values unchanged.

6. Response Validation: Click Next and leave these fields' default settings.

7. Warning & Alert: Verify that the toggle with the label Alerting is turned on.

8. Click Test to check the setup for your uptime check.

9.Press Create. The dashboard page for uptime checks appears when the message "Check and alert created" appears after the create action is successful.

Creating a dashboard and chart

1.Select Monitoring from the Google Cloud console menu  

 Go to Monitoring

2. Choose Dashboards from the navigation bar and then click Create dashboard.

3. Drag the Line chart widget to the graph area from the widget library.

4. Expand the Resource and Metric menu, type CPU in the filter bar, and then choose a particular resource type and metric from the submenus:

• Choose a VM instance from the menu of active resources.
• Select instance from the menu of Active Metric Category options.
• Choose CPU load from the Active metrics menu (1m).
• Tap Apply.

View your logs

Follow these steps to view the logs for the resources shown in a chart:

1. Visit the dashboard where the desired chart is shown.
2. Click more in the chart, then select View logs.

Clean up

Please follow these instructions to prevent your Google Cloud account from being charged for the resources used on this page.

Deleting the project

1.Navigate to the Manage resources page in the Google Cloud console.

Go to Manage resources. 

2. Select the project you wish to delete from the project list, and then click Delete.

3. Enter the project ID in the dialogue box, then click Shut down to destroy the project.

Delete quickstart resource

1.Visit Monitoring

Go to Monitoring

 2. Eliminate the alerting rule:

• Decide on Alerting.
• Click Delete after selecting My Uptime Check Policy.

3. Remove the uptime verification:

• Select for Uptime Checks.
• Click Delete delete after selecting My Uptime.

4. Remove the dashboard:

• Click on Dashboards, then.
• Click Delete to delete the Quickstart dashboard. Verify the deleting process.

5. Get rid of the VM instance you made:

• Navigate to the VM instances page in the Google Cloud console.
• Go to VM instances
• For instance, if you want to delete, check the box.
• Click more to remove the instance. Click on More, then Delete, and then adhere to the directions.

Setting up Datalab to use with Cloud Monitoring.

In this, you'll learn how to configure Datalab for use with your projects for cloud monitoring. You can run ad hoc analyses and visualisations with Datalab's dynamic notebooks that go beyond what Monitoring currently offers.

Setting up Datalab

From a terminal window on your local system, carry out the following actions:

1.Obtain the latest gcloud command:

$ gcloud components update

2. The gcloud datalab component should be installed:

$ gcloud components install datalab

3. Run the following command after changing [PROJECT ID] with your Google Cloud project ID to set up gcloud to connect to your Google Cloud project ID:

$ gcloud config set project [PROJECT_ID]

Run the upcoming command to confirm the configuration:

$ gcloud config get-value project

4. Run the following command to create a Datalab instance, substituting [DATALAB-INSTANCE-NAME] with the name of your instance. All names must start with a lowercase letter, have no more than 62 lowercase letters, numerals, or hyphens, and not include a hyphen at the conclusion.

$ datalab create [DATALAB-INSTANCE-NAME]

5. While the datalab command is in use, the connection to your Datalab instance is still active. The connection is cut off if the terminal command window is closed or stopped. Run the next command to restart the connection:

$ datalab connect [DATALAB-INSTANCE-NAME]

The Datalab Interface

The URL for the newly opened browser window when you run the datalab connect command is http://127.0.0.1:8081/. The Datalab home page is shown on this page:

 

 

Source: Cloud Monitoring

 

You can explore a number of Datalab getting started notebooks, including notebooks for cloud monitoring, in the Datalab docs/ subdirectory.

Monitoring tutorials

There are various interactive Monitoring lessons installed with Datalab:

• Getting Started:

Shows how to integrate the Python Google Cloud's operations suite API into Datalab and specify your default Google Cloud project ID. The API is called using sample code, and monitoring data from your project is obtained.

• Group metrics

Shows how to analyze a project's group structure and how to use groups to aggregate and filter metric data. You must set up a Cloud Monitoring group in order to use group metrics. To learn more, go to Using resource groups.

• Time-Shifted data

Demonstrates some intriguing techniques to modify time series data. The tutorial is set up to potentially use previously-extracted data from a demonstration project because your project might not have enough VM instances to be a decent example.

Running the tutorials

Follow these steps to run the tutorials:

1. In the Datalab interface, select Docs, Tutorials, and Stackdriver Monitoring.
2. Choosing the tutorial. Select the tutorial by clicking its name.
3. Set the project ID to Your Google Cloud project ID should be substituted for my-project-id in the column that contains set datalab project id('my-project-id').
4. Start the tutorial program. Select Run > Run all cells from the menu bar at the top of the screen. This executes all of the tutorial's code again using your current project ID.

Cleaning up

Please follow these instructions to prevent your Google Cloud account from being charged for the resources used on this page.

1. Close any open notebooks by clicking the Running Sessions button in the top-right corner of the Datalab interface. If necessary, you can restart them later. Close any related browser windows or tabs.
2. Close the Datalab tab in your browser by pressing CTRL-C in the Datalab window.
3. From the moment the Datalab VM instance is created until it is deleted, you are responsible for paying charges. After the Datalab VM has been deleted, the Persistent Disk continues to exist until you destroy it.
4. The datalab create command generates extra resources that can be used by other Datalab instances you create.

• Delete the firewall rule that permits SSH access to your Datalab instances, datalab-network-allow-ssh:

 $ datalab delete --delete-disk [DATALAB-INSTANCE-NAME]

• Delete the default connection made by Datalab instances to the datalab-network Virtual Private Cloud (VPC) network:

 $ gcloud compute networks delete datalab-network

• Delete the notebook storage repository you created, datalab-notebooks Cloud Source Repository.

 $ gcloud source repos delete datalab-notebooks

 5. Delete any projects or VM instances you don't wish to maintain from your creations.

Set up a query-based alert by using MQL

In this, you'll learn how to explore metrics, make charts, and set up a query-based alert using Monitoring Query Language (MQL).

Create a Compute Engine instance

Create a Compute Engine instance by completing following steps to have a Google Cloud resource to get metrics from:

1.Choose Compute Engine from the menu on the Google Cloud console.

2. Click Create after selecting Create instance.

Navigate to the Query Editor

Start by performing the following to use MQL:

1. Select Monitoring from the Google Cloud console menu.
2. Choose Metrics Explorer under Monitoring.
3. Choosing the MQL tab.

Perform a simple query and then create a chart

Make a simple query that displays your instance's CPU usage:

1. In the Query Editor, copy the following query and past it

fetch gce_instance::compute.googleapis.com/instance/cpu/utilization
| {
    top 1, max(val())
  ;
    bottom 1, min(val())
  }
| union

2. Select Run Query. 

3. Click Save Chart to store the chart. Give the chart a name and then add it to a dashboard that already exists or create a new one. 

Set up a query-based alert

You can design an alerting strategy that keeps track of a query's outcomes. Complete the following actions to develop such an alerting policy:

1.Pick Monitoring, then select Alerting from the console's menu.

2. Click Edit Notification Channels on the Alerting page. You set up an email address in this stage to receive alert messages.

• Scroll down to Email on the Notification channels page and click Add.
• In the Email Address area, type the email address where you want alert notifications sent.
• In the Display name area, type a precise summary of this channel, such as "Alert on-call."
• Click on Save.
• By clicking Back on the page header, you can return to the Alerting page.

3. Click on Create Policy on the Alerting page.

4. Click Add Condition to specify what you want the alert to be about. The form-based condition editor appears when this action is taken.

• Clicking on code Query Editor will switch to the Query Editor window of the condition editor. The Query Editor and a Configuration pane are included in this version of the condition editor:
• Give your condition a name, e.g., "MQL Quickstart condition."
• Enter the following command to receive a notification when the CPU utilization time for your instance exceeds 2 seconds:

fetch gce_instance::compute.googleapis.com/instance/cpu/usage_time
| window 1m
| condition val() > 2 's'

• To view the chosen data and a threshold line, click Run Query.
• Use the Condition pane as shown to receive an alert when your instance's CPU consumption reaches the threshold and persists there for five minutes:
•  Set the For field's value to five minutes.
•  Keep the value in the Condition triggers if the field is unchanged.
• Click Add to save your condition in your alerting policy.

5. To go to the following step, which is creating notifications, click Next.

6. Click OK after selecting the notification channel you previously established.

7. In order to save your policy, click Save.

Clean up

Follow these instructions to prevent your Google Cloud account from being charged for the resources used on this page.

1.Go to the Google Cloud console's Identity and Access Management page.

Go to the Project Settings

2. After selecting Shut down, proceed as directed.

Frequently asked questions

What is called a cloud?

The term "the cloud" describes the software and databases that run on servers that may be accessed via the Internet.

What is the Cloud Controls Matrix?

The Cloud Controls Matrix (CCM) from the Cloud Security Alliance is a tool created expressly to help potential cloud customers and cloud vendors analyze the overall security risk of a cloud provider.

What is AWS?

AWS (Amazon Web Service)  is an online platform that offers scalable and affordable cloud computing solutions.

What is troubleshooting?

A methodical method of problem-solving known as troubleshooting is frequently used to identify and resolve problems with sophisticated machinery, electronics, computers, and software systems.

For what Compute Engines are used?

The Google infrastructure can be used to build and run virtual machines using Compute Engine, a customized compute service.

Conclusion

This blog has extensively discussed Monitoring a compute engine VM. 

We hope this blog has helped you in enhancing your knowledge about monitoring a compute engine virtual machine. If you want to learn more, check out the excellent content on the Coding Ninjas Website:

Key concepts of cloud logging , Managing the Monitoring Agent ,Incidents in Cloud Monitoring

Refer to our guided paths on the Coding Ninjas Studio platform to learn more about DSA, DBMS, Competitive Programming, Python, Java, JavaScript, etc. 

You can also consider our Online Coding Courses such as the Machine Learning Course to give your career an edge over others.

 

Do upvote our blog to help other ninjas grow. Happy Coding!