Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
Using Cloud Trace with Zipkin
2.1.
Using a container image to set up the server
2.2.
Running the server outside of Google Cloud
2.2.1.
Create or Select a Project
2.2.2.
Create a Service account
2.2.3.
Configure the firewall
2.3.
How to Configure Zipkin Tracers
3.
Creating and viewing analysis reports
3.1.
Daily reports
3.2.
Custom analysis reports
3.3.
Creating a new report
3.4.
Duplicating a report
3.5.
Viewing a report
3.5.1.
Latency pane
3.5.2.
Bottlenecks pane
4.
Setting up Cloud Trace for Java
4.1.
Configuring the stackdriver exporter
4.2.
Enabling full sampling
4.3.
Configure the platform
4.3.1.
Running on Google Cloud
5.
Setting up Cloud Trace for Node.js
5.1.
Installing the client library
5.2.
Configure the client library
6.
Setting up Cloud Trace for Go
6.1.
Installing the package
6.2.
Troubleshooting
6.2.1.
Upload of spans for the Go OpenCensus exporter fails
7.
Frequently Asked Questions
7.1.
What does the histogram present in custom analysis reports display?
7.2.
Which library contains automatic integration for several popular web and RPC frameworks?
7.3.
What command is used to install the Cloud Trace library for Node.js?
8.
Conclusion
Last Updated: Mar 27, 2024

Advanced Concepts of Cloud Trace

Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

Cloud trace is defined as a distributed tracing system that collects latency data from the applications and then displays it on the Google Cloud Console. It automatically analyzes all of the application's traces in order to generate in-depth latency reports for surfacing the application's traces. In this blog, we will cover advanced concepts of Cloud Trace.

Using Cloud Trace with Zipkin

The Zipkin server is used if the application is instrumented with Zipkin and if the user doesn't want to run their own trace backend or wants to access Cloud Trace's advanced analysis tools.

Using a container image to set up the server

GitHub has a container image of the Cloud Trace Zipkin Collector. The repository contains Docker build definition and layers Google Cloud support on the base Zipkin docker image. The user can run this image on their container host of choice, which includes GKE. To run this image, use the below-mentioned command:

$ docker run -d -p 9411:9411 \
-e STORAGE_TYPE=stackdriver \
-e GOOGLE_APPLICATION_CREDENTIALS=/root/.gcp/credentials.json \
-e STACKDRIVER_PROJECT_ID=your_project \
-v $HOME/.gcp:/root/.gcp:ro \
openzipkin/zipkin-gcp


Suppose the user is running this container within the Google Cloud like on a Compute Engine instance. In that case, the environment's default credentials are automatically captured, and traces are automatically sent to the Cloud Trace.

Running the server outside of Google Cloud

The user can build and run the collector outside of the Google Cloud like on a physical server running on-premises by following the below-mentioned steps:

Create or Select a Project

  • If you are a new user, create an account in order to evaluate how the products actually perform in real-world scenarios. When a user creates an account for the first time, the user gets $300 in free credits to run, test, and deploy workloads.
  • Select or create a Google Cloud project from the project selector page available in the Google Cloud Console.

Create a Service account

Follow the below-mentioned steps to allow your service to authenticate to the Cloud Trace API:

  • Create a service account
    • The user should ensure that the new service account has been granted a project editor role so that it has the ability to write data to the Trace API.
    • Click on Furnish a new private key, and choose JSON.
    • Store the JSON credentials file on the machine that is going to run the collector service.

Configure the firewall

The user must configure their network configuration to allow TCP traffic on port 9411 to pass to the machine that is running the Zipkin collector. If the application is hosted outside of the firewall, the user must note that the Zipkin tracer-to-collector traffic is not authenticated. All the connections that are between the Cloud Trace Zipkin collector and the Cloud Trace API are encrypted as well authenticated.

How to Configure Zipkin Tracers

The Zipkin tracers must be configured in order to send data to it irrespective of how the Cloud Trace Zipkin Collector is hosted. The internal IP address, external IP address, or hostname can be used to reference the collector. Each Zipkin tracer needs to be configured differently.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Creating and viewing analysis reports

Daily reports

An automatic report is generated by the trace that is compared with the previous day's performance with the performance from the same day of the previous week for the top three endpoints. The Trace overview page displays the daily report, and it gets listed on the Analysis report page.

Custom analysis reports

As the daily analysis report is created automatically, it doesn't let the user choose the report's contents. A user can create a custom analysis report and choose the traces that are to be included in the report by his/her wish.

The custom reports include the following:

  • Latency data is represented in the form of a histogram.
  • It contains a table that represents a list of the latency by percentile with links to the sample traces.
  • It can also include latency data from different time range for comparison purposes.
  • It lists the RPC calls in a bottleneck pane.


The user can create a new report by mentioning the report parameters manually or by duplicating an existing report and editing the parameters. 

Creating a new report

  • Go to Trace in the Google Cloud Console.
  • Click on New Report from the Analysis reports tab.
  • Choose the following mentioned parameters in the form that appears:
    • Request Filter: Full URL, URL prefix.
    •  HTTP Method: Select a HTTP method for the request, by default, its value is all.
    • HTTP Status: Select a HTTP status that is returned by request, by default, its value is all.
    • Report Name: It is a human-readable name given to the report. If no name is specified, then the specified request filter and the time at which the report was created are used together by trace to create a name.
    • Module: It specifies the App Engine module that handles the request, by default, its value is all.
    • Version: It specifies the version of the App Engine application that handles the request.
  • Select Compare to baseline and enter a time range of the comparison data if you want to compare two sets of latency data.
  • Click on Submit.

Duplicating a report

A trace allows the user to duplicate the filter parameters of an existing report:

  • Go to trace in the Google Cloud Console.
  • Click on the report's name whose structure you want to duplicate from the Analysis report tab.
  • Click on Create similar report.
  • Edit one or more report parameters as per need on the New report request page.
  • Click on Submit

Viewing a report

Follow the below-mentioned steps to view an analysis report:

  • Go to Trace in the Google Cloud Console.
  • Click on the Analysis Reports tab to view the list of system-generated daily reports and custom reports.
  • Click on the Report's name that you wish to view.


Latency pane

It is beneath the graph view and displays latency percentiles along with that, it includes links to traces for representative sample requests. When a trace number is clicked upon, the Trace list page opens and shows detailed information about the root span.

When a comparison report is created, then the latency data pane contains information for both of the data sets.


Bottlenecks pane

The latency value for a span in a specific trace falls outside the expected range of latency values, then it is indicated by a bottleneck. The expected range of the latency values is determined by the Cloud Trace using statistical techniques that rely on computing and comparing empirical latency distributions for subsets of traces. It lists the RPC calls that are significant contributors to latency in the requests when a bottleneck pane is present. The RPC calls are displayed by percentile with representative sample requests.

The bottleneck highlights the RPC calls that cause latency distribution changes between two sets in a comparison report.

Setting up Cloud Trace for Java

Configuring the stackdriver exporter

Use a StackdriverTraceExporter object to export the collected trace data:

public static void createAndRegisterGoogleCloudPlatform(String projectId) throws IOException {
  StackdriverTraceExporter.createAndRegister(
      StackdriverTraceConfiguration.builder().setProjectId(projectId).build());
}


If the user is running on Google Cloud infrastructure, then the user doesn't need to call setProjectID and supply their Google Cloud project ID. But if this field is not set, then the client library for Java automatically gathers the data from the Google Cloud metadata server.

If the user is not running on Google Cloud infrastructure, then the user must supply its google cloud project ID to the application.

When the Google Cloud project ID isn't explicitly set, then the google-cloud Java library automatically determines whether the environment variable GOOGLE_CLOUD_PROJECT is set. If it is set, then the library uses the value of GOOGLE_CLOUD_PROJECT as the Google Cloud project ID>

Enabling full sampling

The sampling rate might be very slow to display the trace data in a developer environment. If you want to sample all the traces, then you can use alwaysSample option.

Use the setSampler method and specify the alwaysSample option to enable the full sampling in a below-mentioned way:

public static void doWorkFullSampled() {
  try (Scope ss =
      tracer
          .spanBuilder("MyChildWorkSpan")
          .setSampler(Samplers.alwaysSample())
          .startScopedSpan()) {
    doInitialWork();
    tracer.getCurrentSpan().addAnnotation("Finished initial work");
    doFinalWork();
  }
} 

Configure the platform

Running on Google Cloud

If the application is running on the google cloud, then the user doesn't need to provide any authentication credentials in the form of a service account to the client library. The user must ensure that the Google Cloud Platform has Cloud trace API access scope enabled.

The default access-scope settings enable the Cloud trace API for the following configurations:

  • App Engine flexible environment
  • App Engine standard environment
  • GKE
  • Compute Engine
  • Cloud Run


If custom access scopes are used, then the user must ensure that the Cloud Trace API access scope is enabled:

  • The gcloud users must specify the access scopes using the –scopes flag and include the trace.append Cloud Trace API access scope. To create a GKE cluster with Cloud Trace API enabled, use the following command:
  • gcloud container clusters create example-cluster-name --scopes=https://www.googleapis.com/auth/trace.append

Setting up Cloud Trace for Node.js

Installing the client library

  • The user must prepare the environment for Node.js development before installing the Cloud Trace library for Node.js
  • Use npm to install the Cloud Trace library for Node.js
  • npm install --save @google-cloud/trace-agent
  • Import the Cloud Trace library for Node.js at the top of the application's main script
  • require('@google-cloud/trace-agent').start();

Configure the client library

The user can customize the behavior of the Cloud Trace library for Node.js. The user can refer to the library's configuration that is available on GitHub for a complete list of configuration options that the user can pass to the library's start method by using an options object.

The below-specified example demonstrates specifying the Google Cloud project ID and setting the path to the credential file. These statements are optional when the application is running on Google Cloud: 

require('@google-cloud/trace-agent').start({
  projectId: 'your-project-id',
  keyFilename: '/path/to/key.json',
});


If the user is running on the Google Cloud infrastructure, then the user doesn't need to set the projectId to their Google Cloud project ID. But if this field is not set, then the client library for Node.js automatically gathers the relevant data from the Google Cloud metadata server.

If the user is not running on the Google Cloud infrastructure, then the user must provide the Google Cloud project ID to the application.

No matter what infrastructure is being used for Node.js, if the Google Cloud project ID is not set explicitly, then the cloud-trace-nodejs library automatically determines if the environment variable GCLOUD_PROJECT is set, and if that's the case, then the library uses the value of GCLOUD_PROJECT as the Google cloud project ID.

Setting up Cloud Trace for Go

Using the OpenCensus, the user can enable Cloud Trace for Go applications. OpenCensus is a set of instrumentation libraries that collects traces and metric data that works with multiple backends.

Installing the package

Using the below-mentioned command, retrieve the OpenCensus trace package:

go get go.opencensus.io/trace

Troubleshooting

Upload of spans for the Go OpenCensus exporter fails

If the bundler library that the exporter uses is unable to manage the number of spans that the application is generating and uploading, then the following types of errors occur:

OpenCensus Stackdriver exporter: failed to upload span: buffer full

OpenCensus Stackdriver exporter: failed to upload 183 spans: buffer full

To resolve such errors, try one or more of the below-mentioned:

  • Increase the value of the TraceSpansBufferMaxBytes field in the Options struct that is passed to the traceExporter constructor. This field specifies the maximum number of bytes that are used for buffering spans, by default, that value is 8MB.
  • Decrease the values for the following fields in the Options object passed to the traceExporter constructor:
    • BundleDelayThreshold: This field specifies the maximum time the exporter can wait before uploading trace spans. By default, this value is set to 2 seconds.
    • BundleCountThreshold: This field specifies the number of spans that can be buffered. By default, this value is set to 50.
  • Try Exporting fewer spans. Configure the global sampler or pass a sampler to each StartSpan to reduce the number of spans you export.

Frequently Asked Questions

What does the histogram present in custom analysis reports display?

The histogram present in the custom analysis reports displays the latency data.

Which library contains automatic integration for several popular web and RPC frameworks?

The OpenCensus library contains automatic integration for several popular web and RPC frameworks.

What command is used to install the Cloud Trace library for Node.js?

To install the Cloud Trace library for Node.js using npm using the following command: npm install --save @google-cloud/trace-agent 

Conclusion

In this article, we have extensively discussed the advanced level concepts of Cloud Trace.

After reading about the advanced level concepts of Cloud Trace,  are you not feeling excited to read/explore more articles on Google Cloud? Don't worry; Coding Ninjas has you covered. To learn about GCP certification: Google Cloud Platform, the difference between AWS, Azure & Google Cloud, and which platform is best: AWS vs. Google Cloud.
Check out this problem - Largest Rectangle in Histogram

If you wish to enhance your skills in Data Structures and AlgorithmsCompetitive ProgrammingJavaScript, etc., you should check out our Guided path column at Coding Ninjas Studio. We at Coding Ninjas Studio organize many contests in which you can participate. You can also prepare for the contests and test your coding skills by giving the mock test series available. In case you have just started the learning process, and your dream is to crack major tech giants like Amazon, Microsoft, etc., then you should check out the most frequently asked problems and the interview experiences of your seniors that will surely help you in landing a job in your dream company. 

Do upvote if you find the blogs helpful.

Happy Learning!

Thank you image
Previous article
Basic Concepts of Cloud Trace
Next article
Cloud Profiler
Live masterclass