Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
Vertex AI Vizier overview
2.1.
Additional Vertex AI Vizier functionality
2.1.1.
Tune parameters
2.1.2.
Optimize any evaluable system
2.2.
How Vertex AI Vizier works
2.2.1.
Study configurations
2.2.2.
Studies and trials
2.2.3.
Measurements
2.2.4.
Search algorithms
2.3.
How Vertex AI Vizier differs from custom training
2.4.
Use cases
3.
Create Vertex AI Vizier studies
3.1.
Before you begin
3.2.
Define constants
3.3.
Construct API requests
3.3.1.
Create a study
3.3.2.
View your study
3.3.3.
 
3.3.4.
Get a study
3.3.5.
List studies
3.3.6.
Evaluate the results
3.3.7.
Add a measurement
3.3.8.
Complete a trial
3.3.9.
List trials
3.3.10.
List optimal trials
4.
Frequently Asked Questions
4.1.
What is a managed dataset in Vertex AI?
4.2.
What is a feature store?
4.3.
How good is Vertex AI?
5.
Conclusion
Last Updated: Mar 27, 2024

Vertex AI Vizier

Author Shivam Sinha
0 upvote
Leveraging ChatGPT - GenAI as a Microsoft Data Expert
Speaker
Prerita Agarwal
Data Specialist @
23 Jul, 2024 @ 01:30 PM

Introduction

In this article, we will discuss Vertex AI Vizier. We will see how it works and how it is different from custom training, we will be discussing Vertex AI Vizer in detail along with its functionality.

Vertex AI Vizier overview

Vertex AI Vizier is a black-box optimization service that can optimize hyperparameters for complex machine learning (ML) models. If your ML model contains different hyperparameters, tuning them manually can be difficult and time-consuming. Vertex AI Vizier optimizes the output of your model by optimizing the hyperparameters.

A black box optimization is an optimization of a system that meets one of the following criteria:

  • No known objective function to evaluate.
  • Systems are usually too complex to be evaluated using an objective function.

Additional Vertex AI Vizier functionality

Vertex AI Vizier optimizes hyperparameters for ML models, but can also perform other optimization tasks.

Tune parameters

Vertex AI Vizier can be used to effectively tune function parameters. For example, a subscribe button on a news site can be used to determine the most effective combination of background color and link color font size. See use cases for more examples.

Read more about the difference between parameters and hyperparameters.

Optimize any evaluable system

Vertex AI Vizier works with any system that can be evaluated, including systems that cannot be expressed as closed-form analytical functions. For example, you can use Vertex AI Vizier to determine the optimal neural network depth, width, and learning rate for your TensorFlow model.

How Vertex AI Vizier works

The following sections define the terms, behaviors, and possible values you can use in Vertex AI Vizier to optimize your Machine Learning model or function. Start by defining your study configuration.

Study configurations

A study configuration is a definition of an optimization problem that you are trying to solve. It contains the results of the optimization and the hyperparameters or parameters that affect the results.

Studies and trials

A study is an implementation of a study configuration. A study uses the goals (metrics) and input values ​​(hyperparameters or parameters) of the study configuration to perform an experiment called a trial. A trial is a set of specific input values ​​that produces measurement results related to a goal.

Vertex AI Vizier suggests input values ​​to use for each experiment but does not run the experiment. The

exam will continue until the specified number of trials is reached or the exam is paused. The trial will continue until you indicate that it is complete or infeasible.

Measurements

The measured value is the measurement result of the experiment. Each measurement can contain one or more metrics, and each experiment can contain one or more measurements collected over a period of time. You can add new measurements to your experiment at any time before you complete the experiment.

Search algorithms

If you don't specify an algorithm, Vertex AI Vizier will use a default algorithm. The default algorithm applies Bayesian optimization to more efficiently search the parameter space to reach the optimal solution.

The following values ​​are available:

  • ALGORITHM_UNSPECIFIED: Same as no algorithm specified. Vertex AI chooses the best search algorithm from Gaussian process bandit, linear combination search, or their variants.
     
  • GRID_SEARCH: Simple grid search within the allowed space. This option is useful for specifying more attempts than the number of points in the space allowed. In such cases, the default algorithm may generate duplicate candidates if no grid search is specified. To use grid search, all parameters must be of type INTEGER, CATEGORICAL, or DISCRETE.
     
  • RANDOM_SEARCH: A simple random search within the allowed range.

How Vertex AI Vizier differs from custom training

Vertex AI Vizier is an independent service for optimizing complex models with different parameters. It can be used for both Machine Learning and non-Machine Learning use cases. It can be used in training jobs or other systems (even multi-cloud). Custom training hyperparameter tuning is a built-in feature that VertexAIVizier uses for training jobs. It helps you determine the best hyperparameter settings for your ML model.

Use cases

Vertex AI helps Vizier fine-tune hyperparameters to optimize models or fine-tune parameters to optimize results in the following scenarios:

  •  Optimize the batch size, learning rate, and other hyperparameters of the neural network recommendation engine.
  • Test different layouts of user interface elements to optimize the usability of your application.
  • Minimize the computational resources of a job by determining the ideal buffer size and number of threads.
  • Optimize the amount of ingredients in recipes to create the tastiest version.

 

Read about Batch Operating System here.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Create Vertex AI Vizier studies

Now, we will discuss how can you make API requests to Vertex AI Vizier by using Python.

Before you begin

  1. If you are visiting the first time to Google Cloud then you need to create an account to evaluate the performance of your Google products in real-world scenarios. New customers will also earn $ 300 in free credit to run, test, and deploy their workloads.
     
  2. Select or create a Google Cloud project in the Google Cloud console, on the project selector page. 
     
  3. Enable the Vertex AI API.
     
  4. Now, you need to create a service account:
    1. In the console, go to the Create service account page.
    2. Select your project.
    3. In the Service account name field, enter a name. The console fills in the Service account ID field based on this name.
    4. In the Service account description field, enter a description. For example, Service accounts for quick start.
    5. Click Create and continue.
    6. To provide access to your project, grant the following role(s) to your service account: Project > Owner.
    7. In the Select a role list, select a role. For additional roles, click add Add another role and add each additional role.
    8. Click Done to finish creating the service account. Do not close your browser window. You will use it in the next step.
       
  5. Create a service account key:In the console, click the email address for the service account that you created.
    1. Click Keys.
    2. Click Add key, and then click Create new key.
    3. Click Create. A JSON key file is downloaded to your computer.
    4. Click Close.
       
  6. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
  • Example: Linux or macOS
  • Example: Windows

 Install the Vertex AI SDK for Python.

Define constants

To define the constants, run the following command, replacing PROJECT_ID and REGION with your project ID and region. Create your own study name or you can also use the suggested values.

import json
import datetime
from google.cloud import aiplatform
REGION = "REGION"
PROJECT_ID = "PROJECT_ID"
# The following placeholder variables are automatically filled in.
STUDY_DISPLAY_NAME = '{}_study_{}'.format(PROJECT_ID.replace('-', ''), datetime.datetime.now().strftime('%Y%m%d_%H%M%S')) #@param {type: 'string'}
ENDPOINT = REGION + '-aiplatform.googleapis.com'
PARENT = 'projects/{}/locations/{}'.format(PROJECT_ID, REGION)

Construct API requests

The following command-line API requests are written in Python.

Create a study

A survey is a series of experiments or trials designed to assist in hyperparameters or parameter optimization.

In these examples, the goal is to find y = x ^ 2 where x is in the range [-10. 10]. This example uses a function that has only one parameter and is easy to calculate to show how to use Vertex AI Vizier.

param_x = {
    'parameter_id': 'x',
    'double_value_spec': {
        'min_value': -10.0,
        'max_value': 10.0
    }
}
metric_y = {
    'metric_id': 'y',
    'goal': 'MAXIMIZE'
}
study = {
    'display_name': STUDY_DISPLAY_NAME,
    'study_spec': {
      'algorithm': 'RANDOM_SEARCH',
      'parameters': [param_x],
      'metrics': [metric_y],
    }
}


To create the study by using your study configuration, send the following request through VizierServiceClient. Use the returned STUDY_NAME to query the study.

vizier_client = aiplatform.gapic.VizierServiceClient(client_options=dict(api_endpoint=ENDPOINT))
study = vizier_client.create_study(parent=PARENT, study=study)
STUDY_NAME = study.name


View your study

After your study is created, you can find the study in the Google Cloud console, in the Vertex AI section, and on the Experiments page.

 

Get a study

To get a study, send the following request.

vizier_client.get_study({'name': STUDY_NAME})


List studies

To list studies in a specific project and region, send the following request:

vizier_client.list_studies({'parent': PARENT})


Evaluate the results

As you receive trial proposals, evaluate each trial and record each result as a measurement.

For example, if the function you want to optimize is y = x^2, evaluate the function at the values ​​of x suggested in your research. If the recommended value is 0.1, the function gives y = 0.1 * 0.1 and gives 0.01.

Add a measurement

After evaluating trial suggestions and obtaining measurements, add the measurements to the trial.

Use the following commands to save measurements and send requests. In this example, replace RESULT with your measurements. If the function to optimize is y = x^2 and the suggested value for x is 0.1, the result is 0.01.

vizier_client.add_trial_measurement({
        'trial_name': TRIAL_ID,
        'measurement': {
            'metrics': [{'metric_id': 'y', 'value':RESULT }]
        }
    })


Complete a trial

After adding all the measurements of the trail, you need to send a command to complete the trail.

It means that when the trial is complete, you can send a command to complete the trial, or send a command to add the final measurement to complete the trial.


Without Final Measurement:

Send the following request to complete a trial without adding a final measurement:

vizier_client.complete_trial({
      'name': TRIAL_ID
  })


With Final Measurement:

To complete a trial and include a final measurement, use the following commands, replacing RESULT with the final measurement.

vizier_client.complete_trial({
      'name': TRIAL_ID
      'final_measurement': {
        'metrics': [{'metric_id': 'y', 'value': RESULT}]
      }
  })

List trials

To list trials in a specific study, send the following request:

vizier_client.list_trials({
    'parent': STUDY_NAME
})


After completing the pending tests, you can call suggestTrials to ask for more suggestions and repeat the test scoring process.

List optimal trials

The following example shows list_optimal_trials. It returns a Pareto optimal trial for a multipurpose study or an optimal trial for a single purpose study.

vizier_client.list_optimal_trials({
    'parent': STUDY_NAME
})

Frequently Asked Questions

What is a managed dataset in Vertex AI?

We recommend using Vertex AI-managed datasets to train your custom models, instead of writing your training application to ingest training data directly. Managed datasets offer the following benefits: Manage your datasets in a central location. Easily create labels and multiple annotation sets.

What is a feature store?

A feature store is a tool for storing commonly used features. When data scientists develop features for a machine learning model, those features can be added to the feature store. This makes those features available for reuse.

How good is Vertex AI?

Vertex AI is a great platform to build some templates for learning machines. We can add a lot of data ( more than 1000GB) and it's scalable. It's very fast to build it, lunch it and compare.

Conclusion

In this article, we discussed Vertex AI Vizier. We also discussed its working and how it is different from custom training .

Refer to our Guided Path on Coding Ninjas Studio to upskill yourself in Data Structures and AlgorithmsCompetitive ProgrammingJavaScriptSystem Design, and many more. If you want to test your competency in programming, you may check out the mock test series and participate in the contests available on Coding Ninjas Studio!

Nevertheless, you may consider our paid courses to give your career an edge over others!

Upvote this blog if you find it helpful and engaging!

Happy Learning!

Live masterclass