Overview of Cloud Tasks

Creating and Adding a task to a Cloud Task Queue

The first topic in the "Overview of Cloud Tasks" series is Adding a task to a Cloud Tasks queue.

Before starting, we have to set up the environment for the system. Here are the brief instructions to set up the environment.

Set up your environment

1. Create or Select a Google Cloud Project. Click on the existing project or create a new one.
    
2. Add an AppEngine application to your project.
    
3. Click on Enable the Cloud Tasks API.
    
4. Now, set up authentication to the API.
    
5. Install and set up the gcloud CLI. You will be able to access the gcloud tool after completing the setup.

Setup the Sample

With the help of these samples, users can test out adding tasks to Cloud Tasks queues using the Cloud Tasks API. These samples provide frameworks like Java, Python, C#, PHP, and many more. We will take the example of python in this section.

The Python sample app has two files. The first is main.py, which is deployed to App Engine as a worker to "process" the task, and the second is  create_app_engine_queue_tasks.py, which is executed locally as a command line tool to create and also add tasks to the queue.

Follow the below instructions to download and install the sample:

Step 1: Copy the sample app repository to your local machine using the given code:

git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

 

Step 2: Go to the directory where the sample code is stored.

cd python-docs-samples/appengine/flexible/tasks/

 

Step 3: Now, install all the dependencies using:

pip install -r requirements.txt

 

Step 4: Ensure the gcloud tool is initialized and set up to use the project you created earlier in the "Overview of Cloud Tasks" series.
 

Step 5: Deploy the worker service on App Engine (main.py):

gcloud app deploy

 

Step 6: Verify that the index page is serving:

gcloud app browse

Create a Cloud Tasks queue

In the environment, you earlier set up, create your queue using the gcloud CLI's gcloud queue management function.

Step 1: Enter the following at the command line.

gcloud tasks queues create [my-queue]

 

Step 2: Wait some time for the queue to initialize, then use "describe" to make sure it was correctly created:

gcloud tasks queues describe my-queue

 

Step 3: Check if your output is matching or not.

name: 
projects/[PROJECT_ID]/locations/[LOCATION_ID]/queues/my-appengine-queue #
Note these ids
rateLimits:
    maxBurstSize: 100
    maxDispatchesPerSecond: 500.0
    maxConcurrentDispatches: 1000
retryConfig:
    maxAttempts: 100
    maxBackoff: 3600s
    maxDoublings: 16
    minBackoff: 0.100s
state: RUNNING

Adding a task to the Cloud Task queue

Step 1: The following environment variables can be manually set on your computer, in the sample app's code, or through an alias. The sample app makes the request that adds tasks to your queue using the following variables:

export PROJECT_ID=PROJECT_ID # The project ID you set up above
export LOCATION_ID=LOCATION_ID # The region in which your queue is running
export QUEUE_ID=my-queue # The queue you created above

Step 2: Make a note that the gcloud command shown below can be used to find the location ID:

gcloud tasks locations list 

 

Step 3: To create a task, we can use local create_app_engine_queue_task.py. Add a hello in the payload section:

python create_app_engine_queue_task.py --project=$PROJECT_ID
--location=$LOCATION_ID --queue=$QUEUE_ID --payload=hello

 

Step 4: Now validate that the payload was received:

gcloud app logs read

Clean Up

Follow these steps to avoid charging your Google Cloud account for the resources used on this page.

1. Go to the Manager Resource in the console.
    
2. Click on the project that you want to delete, and select Delete in the project list.
    
3. Enter the project ID and click on Shut down to delete the project in the dialog box.

HTTP Target tasks

The next topic of the series, "Overview of Cloud Tasks," is HTTP Target tasks. In this section, we will learn how to create HTTP Target tasks. Let's begin.

Cloud Tasks handlers can now be executed on any HTTP endpoint with a public IP address. This includes Cloud Functions, Cloud Run, GKE, Compute Engine, or even an on-prem web server due to this release of HTTP Targets. These services can execute your work in a reliable, configurable way.

Tasks are often created in the form of HTTP requests. The given samples show how to use a service account and the client libraries to manage communication details with the Cloud Tasks server and simplify task creation.

Creating HTTP Target tasks

The tasks in the examples below are created using HTTP Target task requests, which also provide the task handler's URL.

"""Create a task for a user given queue with an arbitrary payload."""


import datetime
import json


from google.cloud import tasks_v2
from google.protobuf import duration_pb2, timestamp_pb2


client = tasks_v2.CloudTasksClient()


# TODO(for developer): Uncomment these lines and replace them with your values.
# queue = 'my-queue'
# project = 'my-project-id'
# location = 'us-central1'
# payload = 'hello' or {'param': 'value'} for json/application
# url = 'https://example.com/task_handler'
# in_seconds = 180
# deadline = 900
# task_name = 'my-unique-task'


# Construct the fully qualified queue name.
parent = client.queue_path(project, location, queue)


# Then, Construct the request body.
task = {
    "http_request": {  # Specify the type of request.
        "http_method": tasks_v2.HttpMethod.POST,
        "url": url,  # Give the full URL path that the task will be sent to.
    }
}
if payload is not None:
    if isinstance(payload, dict):
        # Convert dict to JSON string
        payload = json.dumps(payload)
        # specify HTTP content-type to JSON/application
        task["http_request"]["headers"] = {"Content-type": "application/json"}


    # The payload must be of type bytes, according to the API.
    converted_payload = payload.encode()


    # Now, Add the payload to the request.
    task["http_request"]["body"] = converted_payload


if in_seconds is not None:
    # "seconds from now" should be changed into an rfc3339 datetime string.   
    d = datetime.datetime.utcnow() + datetime.timedelta(seconds=in_seconds)


    # Create timestamp protobuf.
    timestamp = timestamp_pb2.Timestamp()
    timestamp.FromDatetime(d)


    # Add the timestamp to the tasks.
    task["schedule_time"] = timestamp


if task_name is not None:
    # Add the name to tasks.
    task["name"] = client.task_path(project, location, queue, task_name)


if deadline is not None:
    # Then, Add dispatch deadline for requests sent to the worker.
    duration = duration_pb2.Duration()
    task["dispatch_deadline"] = duration.FromSeconds(deadline)


# Now, Use the client to build and send the task.
response = client.create_task(request={"parent": parent, "task": task})


print("Created task {}".format(response.name))

 

The requirement.txt file is 

google-cloud-tasks==2.9.1

Create App Engine tasks

In the series of the "Overview of Cloud Tasks" session, we will now learn how to Create App Engine tasks and then place them in Cloud Tasks queues. When using this procedure, you may optionally give task-specific data to the handler and explicitly specify the service and handler that should handle the task.

Tasks are created in the form of an HTTP request that you can design however you like. You can manage the details of low-level communication with the server, including Google authentication, using the client libraries, as the following samples do.

"""Create a task for a user given queue with an arbitrary payload."""


from google.cloud import tasks_v2
from google.protobuf import timestamp_pb2
import datetime
import json


# Create a client.
client = tasks_v2.CloudTasksClient()


# TODO(for developer): Uncomment these lines and replace them with your values.
# project = 'my-project-id'
# queue = 'my-appengine-queue'
# location = 'us-central1'
# payload = 'hello' or {'param': 'value'} for application/json
# in_seconds = None


# Construct the fully qualified queue name.
parent = client.queue_path(project, location, queue)


# Construct the request body.
task = {
    'app_engine_http_request': {  # Specify the type of request.
        'http_method': tasks_v2.HttpMethod.POST,
        'relative_uri': '/example_task_handler'
    }
}
if payload is not None:
    if isinstance(payload, dict):
        # Convert dict to JSON string
        payload = json.dumps(payload)
        # specify HTTP content-type to application/JSON
        task["app_engine_http_request"]["headers"] = {
            "Content-type": "application/json"
        }
        # The payload must be of type bytes, according to the API.
    converted_payload = payload.encode()


    # Add the payload to the request.
    task['app_engine_http_request']['body'] = converted_payload


if in_seconds is not None:
    # "seconds from now" should be changed into an rfc3339 datetime string.
    d = datetime.datetime.now(tz=datetime.timezone.utc) + datetime.timedelta(
        seconds=in_seconds)


    # Create a Timestamp protobuf.
    timestamp = timestamp_pb2.Timestamp()
    timestamp.FromDatetime(d)


    # Now, Add the timestamp to the tasks.
    task['schedule_time'] = timestamp


# Then, Use the client to build and send the task.
response = client.create_task(parent=parent, task=task)


print('Created task {}'.format(response.name))
return response

 

The requirement.txt file is:

Flask==2.1.0; python_version > '3.6'
Flask==2.0.3; python_version < '3.7'
gunicorn==20.1.0
google-cloud-tasks==2.7.1

Configure the Cloud Tasks queue

In the "Overview of Cloud Tasks" series, we will now understand how to configure the Cloud Tasks queue.

When you configure a Cloud Tasks queue, all the tasks in that queue will use that configuration, regardless of when you do it. 

The three aspects of configuring your queues are:

1. Routing: The name and version of the service containing the appropriate worker are known to the queue. The process is called target.
    
2. Rate limit: You can set the number of concurrent tasks and the maximum rate that can be dispatched by a queue.
    
3. Retry parameters: When a task fails to complete, Cloud Tasks will retry it with an exponential backoff based on the parameters you specified.

Manage queues and tasks

Now we will understand how to manage queues and tasks like deleting or pausing queues in the series of "Overview of Cloud Tasks."

Deleting a task from a queue

Follow the steps to delete a single task using the console:

1. In the console, open the Cloud Tasks page.
    
2. Find the task that you want to delete and click on the queue's name.
    
3. Select the task and click on Delete task.
    
4. Click Delete.

Pausing queues

Moving ahead in the "Overview of Cloud Tasks" series, the next topic is Pausing queues.

Follow the steps to pause or resume a queue using the console:

1. In the console, open the Cloud Tasks queues page.
    
2. Click on the Pause queue after selecting the name of the queue you want to pause. 
   You can also resume the queue by clicking on the Resume queue.
    
3.  Confirm your action.

Frequently Asked Questions

What is Cloud Run GCP?

A managed Compute Platform called Cloud Run lets you run containers that may be invoked through requests or events. As Cloud Run is serverless, you can focus on what matters: creating excellent applications. It abstracts away any infrastructure maintenance.

What is IAM in GCP?

IAM (Identity and Access Management) gives administrators complete power and visibility to manage Google Cloud resources centrally by allowing them to approve who can execute actions on particular resources.

Name the methods that come in google.cloud.location.Locations.

There are mainly two methods that come inside google.cloud.location.Locations:

• GetLocation: It is used to gather information about a location.
   
•  ListLocations: It makes the list of information about the supported location.

Conclusion

We have discussed the topic of Overview of Cloud Tasks. We have seen how to work on a cloud task, such as creating tasks, adding a task, and many more.

We hope this blog has helped you enhance your knowledge of "Overview of Cloud Tasks." If you want to learn more, check out our articles Cloud ERP, Cluster computing, Overview of a log based metric, and many more on our platform Coding Ninjas Studio.

Check out this problem - Queue Implementation

But suppose you have just started your learning process and are looking for questions from tech giants like Amazon, Microsoft, Uber, etc. In that case, you must look at the problems, interview experiences, and interview bundle for placement preparations.

However, you may consider our paid courses to give your career an edge over others!

Happy Learning!