Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
gcloud eventarc
3.
Develop Event Receiver
3.1.
Event Receiver Response
3.2.
Open Source
3.3.
Using a CloudEvents SDK library
3.4.
Sample Receiver Source Code
4.
Receive events using the Cloud Audit Logs (gcloud CLI)
4.1.
Create a Cloud Storage Bucket
4.2.
Deploy the event receiver service to Cloud Run
4.3.
Create an Eventarc trigger
4.4.
Generating and Viewing an Event
5.
Receive events using Pub/Sub messages
5.1.
Set up a Google Service Account
5.2.
Enable GKE Destinations 
5.3.
Create a Pub/Sub Trigger
5.4.
Generate and View an Event
5.5.
Clean Up
6.
Migrate Pub/Sub Triggers
6.1.
Enable GKE Destinations
6.2.
Enable Workload Identity on a cluster
6.3.
Identifying existing Events for Cloud Run for Anthos triggers
6.4.
Create an Eventarc trigger to replace the existing trigger
6.5.
Clean up after Migration
7.
Frequently Asked Questions
7.1.
What is GCP Eventarc?
7.2.
What is Cloud Run GCP?
7.3.
Is Cloud Run a PaaS?
8.
Conclusion
Last Updated: Mar 27, 2024
Medium

Overview of Eventarc

Author Sagar Mishra
1 upvote

Introduction

Eventarc is a new eventing functionality that allows you to submit events from more than 60 Google Cloud sources to Cloud Run. It operates by reading Audit Logs from multiple sources and transmitting them as events in the CloudEvents format to Cloud Run services. It can also read events from Pub/Sub topics for customized applications.

Google Cloud Platform

This article will discuss the Overview of Eventarc, like Receiving events using the Cloud Audit Logs. Also, Developing Event Receiver, Receiving events using Pub/Sub messages, and Migrating Pub/Sub Triggers.

gcloud eventarc

The main function of gcloud eventarc is to manage Eventarc resources
 

GCLOUD WIDE FLAGS

All commands have access to these flags: --help.

For details, run the following:

$ gcloud help

GROUP

  • audit-logs-provider: Examine the provider serviceNames and methodNames for the Eventarc event type google.cloud.audit.log.v1.written.
     
  • channel-connections: It is used to manage Eventarc channel connections.
     
  • channels: It is used to manage Eventarc channels.
     
  • google-channels: It is used to manage Eventarc Google channels.
     
  • locations: It is used to explore locations available for Eventarc.
     
  • providers: It is used to explore event providers available in Eventarc.
     
  • triggers: It is used to manage Eventarc triggers.

Develop Event Receiver

This section in the "Overview of Eventarc" series will deal with the topic Development of Event Receiver.

The following event types can be obtained via event providers (sources):

  • Cloud Audit
  • Direct events
  • Messages published to Pub/Sub topics

Event Receiver Response

To inform the router that an event was successfully received, your receiver service should deliver an HTTP 2xx response. The router will resend the event and considers any other HTTP responses to be delivery failures.

Open Source

The CloudEvents GitHub repository contains the HTTP body's structure for every event.

The repository includes the following to help you understand and use CloudEvents data in your programming language:

  1. Google Protocol Buffers for CloudEvents data payloads
  2. Generate JSON schemas
  3. A public JSON schema catalog

Using a CloudEvents SDK library

CloudEvents SDK library is used to develop event receiver services. This feature is available in the given languages:

  • C#
  • Go
  • Java
  • Ruby
  • JavaScript
  • Python

Sample Receiver Source Code

You can read Cloud Storage events using Cloud Audit Logs. It is available in a service deployed to Cloud Run. The sample code is below.

@app.route('/', methods=['POST'])
def index():
    # Gets the GCS bucket name from the CloudEvent header
    # Example: "storage.googleapis.com/projects/_/buckets/my-bucket"
    bucket = request.headers.get('ce-subject')


    print(f"Detected change in Cloud Storage bucket: {bucket}")
    return (f"Detected change in Cloud Storage bucket: {bucket}", 200)
You can also try this code with Online Python Compiler
Run Code

 

To read Pub/Sub events in a service deployed to Cloud Run, see the code below:

@app.route('/', methods=['POST'])
def index():
    data = request.get_json()
    if not data:
        msg = 'no Pub/Sub message received'
        print(f'error: {msg}')
        return f'Bad Request: {msg}', 400


    if not isinstance(data, dict) or 'message' not in data:
        msg = 'invalid Pub/Sub message format'
        print(f'error: {msg}')
        return f'Bad Request: {msg}', 400


    pubsub_message = data['message']


    name = 'World'
    if isinstance(pubsub_message, dict) and 'data' in pubsub_message:
        name = base64.b64decode(pubsub_message['data']).decode('utf-8').strip()


    resp = f"Hello, {name}! ID: {request.headers.get('ce-id')}"
    print(resp)


    return (resp, 200)
You can also try this code with Online Python Compiler
Run Code

Receive events using the Cloud Audit Logs (gcloud CLI)

In the series of "Overview of Eventarc," our first topic is how to receive events from the Cloud Storage in an unauthenticated Cloud Run service using Eventarc. 

Step 1: Make sure you have an account in Google Cloud; if not, create one
 

Step 2: Select or create a Google Cloud project on the project selector page in the Google cloud console.
 

Step 3: Ensure that the billing is enabled for your project. You can check if billing is enabled on your project also.
 

Step 4Enable the APIs like Cloud Run, Cloud Logging, Eventarc, Pub/Sub, and Cloud Build.
 

Step 5: Update the gcloud components using the below code:

gcloud components update

 

Step 6: Log in to your account.

gcloud auth login

 

Step 7: Set the configuration variables:

gcloud config set project PROJECT_ID
gcloud config set run/region us-central1
gcloud config set run/platform managed
gcloud config set eventarc/location us-central1
You can also try this code with Online Python Compiler
Run Code

 

Step 8: In Google Cloud Storage, Enable the following Cloud Audit Log:

  1. Admin Read
  2. Data Read
  3. Data Write
     

Step 9: Give the Compute Engine service account the eventarc.eventReceiver role:

gcloud projects add-iam-policy-binding PROJECT_ID \    --member=serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \
    --role='roles/eventarc.eventReceiver'

 

Step 10: Install the Git source code management tool after downloading.

Create a Cloud Storage Bucket

We can create a Storage bucket simply by using:

gsutil mb -l us-central1 gs://events-quickstart-PROJECT_ID/

After the complete event source is created, you can also deploy the event receiver service on Cloud Run.

Deploy the event receiver service to Cloud Run

Set up a Cloud Run service to receive and log events. The samples provide frameworks like Java, Python, C#, PHP, and many more. We will take the example of python in this section. To put the sample event receiver service into operation:

Step 1Clone the repository

git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
cd python-docs-samples/eventarc/audit-storage

 

Step 2Upload it to Cloud Builder after building the container.

gcloud builds submit --tag gcr.io/PROJECT_ID/helloworld-events

 

Step 3Deploy the container image to Cloud Run.

gcloud run deploy helloworld-events \
  --image gcr.io/PROJECT_ID/helloworld-events \
    --allow-unauthenticated


The command line will then display the service URL after the successful deployment.

Create an Eventarc trigger

In this section of the series "Overview of Eventarc," we will learn to create an Eventarc trigger. The Eventarc is used to send events from the Cloud Storage bucket to the helloworld-events Cloud Run service.

Step 1Create a trigger that uses the Google Cloud project default service account and filters Cloud Storage events:

gcloud eventarc triggers create events-quickstart-trigger \
      --destination-run-service=helloworld-events \
      --destination-run-region=us-central1 \
      --event-filters="type=google.cloud.audit.log.v1.written" \
      --event-filters="serviceName=storage.googleapis.com" \
      --event-filters="methodName=storage.objects.create" \
  	  --service-account=PROJECT_NUMBER-compute@developer.gserviceaccount.com
You can also try this code with Online Python Compiler
Run Code


This will create a trigger known as events-quickstart-trigger.
 

Step 2: Run the command given below to confirm if the events-quickstart-trigger was successfully created.

gcloud eventarc triggers list --location=us-central1

Generating and Viewing an Event

Step 1Upload the text file to a Cloud Storage to generate an Event:

echo "Hello World" > random.txt
gsutil cp random.txt gs://events-quickstart-PROJECT_ID/random.txt


The Cloud Run service logs the event's message once the upload creates an event.

 

Step 2To view the log entries, your service has made for events:

gcloud logging read "resource.type=cloud_run_revision AND resource.labels.service_name=helloworld-events AND NOT resource.type=build"

 

We have completed a Cloud Audit in the first section of the "Overview of Eventarc" series. This includes Creating a Cloud Bucket, an event receiver service to Cloud Run, Creating an Eventarc, and Generating and Viewing an Event.

Receive events using Pub/Sub messages

In the series of "Overview of Eventarc," we will now discuss how to receive events using Pub/Sub messages stepwise.

If you are new to the Google Cloud, then create a new account. Ensure that the billing is enabled for your Cloud Project. Then install and initialize the Google Cloud CLI.

Set up a Google Service Account

Create a user-provided service account and assign it a few particular roles. This will enable the event forwarder component to retrieve events from Pub/Sub.

Step 1Create a service account, also known as TRIGGER_GSA that is used to create triggers:

TRIGGER_GSA=eventarc-gke-triggers
gcloud iam service-accounts create $TRIGGER_GSA

 

Step 2Grant the monitoring.metricWriter and pubsub.subscriber roles to the service account:

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/pubsub.subscriber"
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/monitoring.metricWriter"
You can also try this code with Online Python Compiler
Run Code

Enable GKE Destinations 

We can use a prebuilt image, gcr.io/cloudrun/hello, to deploy a GKE service that will receive and log events:

Step 1: Now create a Kubernetes deployment:

SERVICE_NAME=hello-gke
kubectl create deployment $SERVICE_NAME \
--image=gcr.io/cloudrun/hello

 

Step 2Then, Expose it as a Kubernetes service:

kubectl expose deployment $SERVICE_NAME \
--type LoadBalancer --port 80 --target-port 8080

Create a Pub/Sub Trigger

The Eventarc trigger sends some messages to the hello-gke GKE service whenever a message is published to the Pub/Sub topic.

Step 1To listen for Pub/Sub messages, create a GKE trigger. The below code is for the new Pub/Sub topic:

gcloud eventarc triggers create gke-trigger-pubsub \
  --destination-gke-cluster=events-cluster \
  --destination-gke-location=us-central1 \
  --destination-gke-namespace=default \
  --destination-gke-service=hello-gke \
  --destination-gke-path=/ \
  --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
  --service-account=$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com

In case you are an existing user, then use the below code:

gcloud eventarc triggers create gke-trigger-pubsub \
  --destination-gke-cluster=events-cluster \
  --destination-gke-location=us-central1 \
  --destination-gke-namespace=default \
  --destination-gke-service=hello-gke \
  --destination-gke-path=/ \
  --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
  --service-account=$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com \
  --transport-topic=projects/PROJECT_ID/topics/TOPIC_ID
You can also try this code with Online Python Compiler
Run Code

 

  • Give your Google Cloud project ID in place of PROJECT_ID.
  • Give ID of the existing Pub/Sub topic in TOPIC_ID.
     

Step 2Make sure that the trigger was created successfully:

gcloud eventarc triggers list


The output should look like this:

NAME: gke-trigger-pubsub
TYPE: google.cloud.pubsub.topic.v1.messagePublished
DESTINATION: GKE: hello-gke
ACTIVE: Yes

Generate and View an Event

Publishing a text message to the Pub/Sub topic will trigger an event to be created that will start the GKE service. After that, you can see the message in the pod logs.

Step 1As an environment variable, find and set the Pub/Sub topic:

TOPIC=$(gcloud eventarc triggers describe gke-trigger-pubsub --format='value(transport.pubsub.topic)')

 

Step 2To generate an event, send a message to the Pub/Sub

gcloud pubsub topics publish $TOPIC --message="Hello World"

 

Step 3To view the event message, use the below code:

a. Find the pod Id:

kubectl get pods


The output screen should look like the following:

 

b. Check the log of the pod:

kubectl logs NAME


Note: Replace NAME with the copied name of the pod.
 

c. Look for a log entry matching:

2022/02/24 22:23:49 Hello from Cloud Run! The container started successfully and is listening for HTTP requests on $PORT
{"severity":"INFO","eventType":"google.cloud.pubsub.topic.v1.messagePublished","message":"Received event of type google.cloud.pubsub.topic.v1.messagePublished. Event data: Hello World"[...]}

Clean Up

Even if Cloud Run does not charge when the service is not in use, there is a chance that you will be paid. For instance, the Container Registry contains the GKE clusterEventarc resourcesPub/Sub messages, and container image storage.

Migrate Pub/Sub Triggers

In the series of "Overview of Eventarc," our last topic is to migrate Pub/Sub triggers.

Eventarc can be used to migrate event triggers so that a Cloud Run for Anthos service can accept Pub/Sub events. This section of the "Overview of Eventarc" series assumes your Cloud Run for Anthos service runs in a Google Kubernetes Engine (GKE) cluster. And also, you are migrating existing Events for Cloud Run for Anthos triggers. Each cluster needs to have a migration run.

The migration path minimizes event duplication while avoiding event loss. The process of migration includes the following steps:

Enable GKE Destinations

Enable GKE destinations and assign the key role to the Eventarc service account to enable Eventarc to manage resources in the GKE cluster.

Step 1For Eventarc, enable GKE.

gcloud eventarc gke-destinations init

 

Step 2Enter y to bind the required role at the prompt. The following roles are bound:

  • roles/compute.viewer
  • roles/container.developer
  • roles/iam.serviceAccountAdmin

Enable Workload Identity on a cluster

You can skip this step if your Workload identity is already enabled on your cluster.

The suggested way for using applications running on Google Kubernetes Engine (GKE) to access Google Cloud services. It is preferred because of its enhanced manageability and security attributes. Also, utilizing Eventarc, Cloud Run for Anthos events must be forwarded.

Refer to Using Workload Identity to enable Workload Identity on an existing cluster,

Identifying existing Events for Cloud Run for Anthos triggers

Before migrating any existing Events for the Cloud Run for Anthos triggers, you have to retrieve the trigger details.

Step 1List the cluster's current Events for Cloud Run for Anthos triggers:

gcloud beta events triggers list --namespace EVENTS_NAMESPACE


The namespace of your events broker should be used in place of EVENTS_NAMESPACE.

The output screen should look like the following:



 

Step 2To create the Eventarc trigger, you must have the subject ID. Get the topic ID for the existing Cloud Run for Anthos trigger events:

gcloud beta events triggers describe TRIGGER_ID \
--platform gke --namespace EVENTS_NAMESPACE \
--format="flattened(serialized_source.spec.topic,serialized_trigger.spec.filter.attributes.type,serialized_trigger.spec.subscriber.ref.name,serialized_trigger.spec.subscriber.ref.namespace)"

 

Step 3: Replace TRIGGER_ID with a fully qualified identification or the existing Events for Cloud Run ID for Anthos trigger.

The output screen should look like the following:

Create an Eventarc trigger to replace the existing trigger

Now, Set up a user-managed service account and grant it particular roles before creating the Eventarc trigger. Do this so that Eventarc can manage events for Cloud Run for Anthos destinations.

Step 1Create a GSA (Google Cloud Service Account):

TRIGGER_GSA=SERVICE_ACCOUNT_NAME
gcloud iam service-accounts create $TRIGGER_GSA

 

Grant the monitoring.metricWriter and pubsub.subscriber roles to the service account:
 

PROJECT_ID=$(gcloud config get-value project)
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/pubsub.subscriber"
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/monitoring.metricWriter"

 

Step 2Based on the setup of the existing Events for Cloud Run for Anthos trigger, create a new Eventarc trigger. The arguments should all be identical to the existing Events for Cloud Run for Anthos trigger. It includes the destination service, cluster, and subject ID.

gcloud eventarc triggers create EVENTARC_TRIGGER_NAME \
  --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
  --location LOCATION \
  --destination-gke-service=DESTINATION_SERVICE \
  --destination-gke-cluster=CLUSTER_NAME  \
  --destination-gke-location=CLUSTER_LOCATION  \
  --destination-gke-namespace=EVENTS_NAMESPACE  \
  --destination-gke-path=/ \
  --service-account=$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com \
  --transport-topic=projects/PROJECT_ID/topics/TOPIC_ID
You can also try this code with Online Python Compiler
Run Code


Replace TOPIC_ID with the Pub/Sub topic ID that you previously retrieved and EVENTARC TRIGGER_NAME with the name for the new Eventarc trigger.
 

Step 3Confirm that trigger was created successfully:

gcloud eventarc triggers list
You can also try this code with Online Python Compiler
Run Code

Clean up after Migration

You can delete or remove the original Events for Cloud Run for Anthos trigger after testing and confirming the trigger's migration to Eventarc.
 

Delete the Events for Cloud Run for Anthos trigger:

gcloud beta events triggers delete TRIGGER_NAME \
  --platform gke \
  --namespace EVENTS_NAMESPACE \
  --quiet
You can also try this code with Online Python Compiler
Run Code


An existing Events for Cloud Run for Anthos trigger has been transferred to Eventarc.

Frequently Asked Questions

What is GCP Eventarc?

Eventarc allows you to deliver events from Google services, SaaS. And also allow your apps to use loosely coupled services that react to state changes. Eventarc needs no infrastructure control. You can optimize productivity. And costs while building a modern, event-driven solution.

What is Cloud Run GCP?

A Managed Compute Platform called Cloud Run lets you run containers that may be invoked through requests or events. Because Cloud Run is serverless, you can focus on what matters, like creating excellent applications. It abstracts away any infrastructure upkeep.

Is Cloud Run a PaaS?

The PaaS architecture called Cloud Run manages the entire container lifetime. That method will be applied in this code lab. You can bring your clusters and Pods from GKE by using Cloud Run for Anthos, which is Cloud Run with an extra layer of control.

Conclusion

We have discussed the topic of Overview of Eventarc. We have seen how to receive events using the Cloud Audit, such as creating a bucket, deployment, and many more.

We hope this blog has helped you enhance your knowledge of "What is Hibernate Caching." If you want to learn more, check out our articles Overview of Azure App Service Local CacheXaas in Cloud ComputingCluster Computing, and many more on our platform Coding Ninjas Studio.

But suppose you have just started your learning process and are looking for questions from tech giants like Amazon, Microsoft, Uber, etc. In that case, you must look at the problemsinterview experiences, and interview bundle for placement preparations.

However, you may consider our paid courses to give your career an edge over others!

Happy Learning!

Live masterclass