Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Last Updated: Mar 27, 2024
Difficulty: Medium

Vertex AI TensorBoard

Leveraging ChatGPT - GenAI as a Microsoft Data Expert
Speaker
Prerita Agarwal
Data Specialist @
23 Jul, 2024 @ 01:30 PM

Introduction

In machine learning, in order to improve something, we often need to be able to measure it. TensorBoard is a tool that provides the necessary measurements and visualizations during machine learning workflows. This allows tracking of experimental metrics such as loss and accuracy, visualization of model plots, and projection of embeddings ontolow-dimensional space.

Vertex AI TensorBoard

Overview

Open root TensorBoard( TB) is a Google open root plan for machine knowledge test visualization. Vertex AI TensorBoard is a company-prepared managed performance of TensorBoard.

Vertex AI TensorBoard provides multicolored detailed visualizations, that include:

  • Tracking and picturing criteria like loss and delicacy over time
  • Visualizing model computational graphs( ops and layers)
  • examining histograms of weights, impulses, or other tensors, as they change over time
  • Projecting embeddings to a lower dimensional space
  • Displaying image, manual, and audio samples
A Vertex AI TensorBoard alternative for smaller budgets (Part 2) | by Tim  De Smet | ML6team

In addition to the heavy visualizations from TensorBoard, Vertex AI TensorBoard provides:

  • An insistent, shareable link to your experiment's dashboard
  • A searchable list of all experiments in a system
  • Tight integrations with Vertex AI services for model training
  • Enterprise-grade security, separateness, and deference

With Vertex AI TensorBoard, you can track, picture, and compare ML tests and participate in them with your outfit.

Get Started

Before we can visualize our experiment, we need to create a Vertex AI TensorBoard instance. This is a localized resource that stores your Vertex AI TensorBoard experiments. You can create multiple instances within your project

Create a Vertex AI TensorBoard Instance

Follow the following steps to produce a Vertex AI TensorBoard instance with console.

  1. Set up your design and environment, If you're new to Vertex AI or starting a new design.
  2. Go to the experiment page in the Vertex AI section of the Google Cloud console. Go to the Experiment page
  3. Navigate to the TensorBoard Instances tab.
  4. Click Create Option at the top of the page.
  5. Select a region from the drop-down list.
  6. Click Create button to create your TensorBoard instance.
     

You can also make a Vertex AI TensorBoard case from the Google Cloud CLI.

  1. Install the gcloud CLI.
     
  2. Initialize the Google Cloud CLI by running gcloud init.
     
  3. Explore the commands to confirm the installation.
gcloud ai tensorboards-- help


The commands include create,  list, describe, and delete. However, you can follow this way to set default values for your design and region before proceeding.
Now, you need to create a Vertex AI TensorBoard instance.

4. Authenticate to the gcloud CLI.


gcloud auth application-default login


5. Create a Vertex AI TensorBoard instance by handing a project name and a display name. This step may take few minutes to complete for the first time in the project. Mark the Vertex AI TensorBoard instance name.

gcloud ai tensorboards create -- display- name DISPLAY_NAME
– Project PROJECT_NAME


Replace the following:

PROJECT_NAME the system in which you want to create the TensorBoard instance
 

DISPLAY_NAME: an illustrative name for the TensorBoard instance.
 

Uploading TensorBoard logs with the Vertex AI TensorBoard Uploader

Vertex AI TensorBoard offers a Python CLI for uploading TensorBoard logs. You can upload logs from any environment which can connect to Google Cloud.

You can also upload TensorBoard logs using the Vertex AI TensorBoard REST APIs.

Install the Vertex AI TensorBoard Uploader through Vertex AI SDK

The uploader needs the closing reading of pip in order to get installed right.

pip install -U pip
pip install google-cloud-aiplatform[tensorboard]


Uploading Vertex AI TensorBoard logs

tb-gcp-uploader --tensorboard_resource_name \
  TENSORBOARD_INSTANCE_NAME \
  --logdir=LOG_DIR \
  --experiment_name=TB_EXPERIMENT_NAME --one_shot=True


TENSORBOARD_INSTANCE_NAME: We can identify the instance name in two ways:

  • The full name is printed at the end of the gcloud ai tensorboards create command that you used previously.

If the TensorBoard Instance was created using the console, the TENSORBOARD_INSTANCE_NAME is

 projects/PROJECT_ID_OR_NUMBER/locations/REGION/tensorboards/TENSORBOARD_INSTANCE_ID

  • Visit the Experiments page Vertex AI section of the Google Cloud console, to find the TENSORBOARD_INSTANCE_ID and then you need to select the TensorBoard Instances tab.
  • Visit Experiments page
  • LOG_DIR: the location of the event logs that resides either in the local file system or Cloud Storage
  • TB_EXPERIMENT_NAME: the name of the experiment, for example test-experiment. The experiment name should be unique within a TensorBoard resource. The uploader CLI by default runs indefinitely, monitoring changes in the LOG_DIR, and uploads newly added logs. --one_shot=True disables the behavior. Run tb-gcp-uploader --help for more information.

View a Vertex AI TensorBoard experiment

To access the Vertex AI TensorBoard user interface, you need the Vertex AI TensorBoard Web App User IAM role that your project's IAM administrator can grant. It can also be accessed by users with the Vertex AI admin role.

Use the console

Follow the below steps to view your Vertex AI TensorBoard experiment from the console.

  • Go to the Experiments page, In the Vertex AI section of the console.
  • Filter or scroll the experiments list to find your experiment from the experiments tab.
  • Click Open TensorBoard to open the Vertex AI TensorBoard UI which is next to the experiment.
     

Or, if you're using Vertex AI TensorBoard with custom training, select a training job on the Training page.

Use Vertex AI TensorBoard with custom training

If you use custom training to train your model, you can configure your training job to automatically upload Vertex AI TensorBoard logs to Vertex AI TensorBoard. For existing Vertex AI TensorBoard users, no changes to the model training code are required.

With this integration, you can monitor your training in near real-time by streaming as they are written to Cloud Storage.

For using the Vertex AI TensorBoard with custom training, follow these steps:

Create a service account with the required permissions

The Vertex AI TensorBoard integration with custom training requires attaching a service account.

  1. Create a service account by following the command:
gcloud --project=PROJECT_NAME iam service-accounts create USER_SA_NAME


  Where,

  • PROJECT_NAME: project's name in which service account is created
  • USER_SA_NAME: a unique name for the service account


2. Vertex AI Training Service will use the new service account to access Google Cloud services and resources. Assign these roles as needed using the following command:

SA_EMAIL="USER_SA_NAME@PROJECT_NAME.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding PROJECT_NAME \
   --member="serviceAccount:${SA_EMAIL}" \
   --role="roles/storage.admin"

gcloud projects add-iam-policy-binding PROJECT_NAME \
   --member="serviceAccount:${SA_EMAIL}" \
   --role="roles/aiplatform.user"

View your Vertex AI TensorBoard in the console

The Vertex AI TensorBoard Web App User IAM role is required to access the Vertex AI TensorBoard user interface. This can be granted by the project's IAM administrator. Users with the VertexAI Administrator role can also access it. 

To view the VertexAITensorBoard associated with the training job, 

  • Go to the training page in the VertexAI section of the console.
  • Select the Custom Jobs tab.
  • Select a training job to view the training details page.
  • At the top of the page, Click the open TensorBoard button
Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Frequently Asked Questions

How good is Vertex AI?

Vertex AI is a great platform to build some templates for learning machines. We can add a lot of data ( more than 1000GB) and it's scalable. It's very fast to build it, lunch it and compare.

Is Vertex AI replacing the AI platform?

Vertex AI replaces legacy services such as AI Platform Data Labeling, AI Platform Training, and Prediction, AutoML Natural Language, AutoML Video, AutoML Vision, AutoML Tables, and AI Platform Deep Learning Containers

What is a managed dataset in Vertex AI?

We recommend using Vertex AI-managed datasets to train your custom models, instead of writing your training application to ingest training data directly. Managed datasets offer the following benefits: Manage your datasets in a central location. Easily create labels and multiple annotation sets.

Conclusion

In this article, we discussed Vertex AI TensorBoard. We also discussed how to create, view, and use a Vertex AI TensorBoard.

Refer to our Guided Path on Coding Ninjas Studio to upskill yourself in Data Structures and AlgorithmsCompetitive ProgrammingJavaScriptSystem Design, and many more. If you want to test your competency in programming, you may check out the mock test series and participate in the contests available on Coding Ninjas Studio!

Nevertheless, you may consider our paid courses to give your career an edge over others!

Upvote this blog if you find it helpful and engaging!

Happy Learning!

thank you image

 

Topics covered
1.
Introduction
2.
Vertex AI TensorBoard
2.1.
Overview
2.2.
Get Started
2.2.1.
Create a Vertex AI TensorBoard Instance
2.2.2.
Uploading TensorBoard logs with the Vertex AI TensorBoard Uploader
2.2.3.
Install the Vertex AI TensorBoard Uploader through Vertex AI SDK
2.2.4.
Uploading Vertex AI TensorBoard logs
2.3.
View a Vertex AI TensorBoard experiment
2.3.1.
Use the console
2.4.
Use Vertex AI TensorBoard with custom training
2.4.1.
Create a service account with the required permissions
2.5.
View your Vertex AI TensorBoard in the console
3.
Frequently Asked Questions
3.1.
How good is Vertex AI?
3.2.
Is Vertex AI replacing the AI platform?
3.3.
What is a managed dataset in Vertex AI?
4.
Conclusion