Pub/Sub: Push Subscriptions and Webhooks

Step 0: Authentication

How this works: You need a Google account that uses the same email address as your GCP Study Hub account. When you sign into that Google account, the system will be able to grant you temporary access to a GCP project when you start the lab.

Click the button below to start your lab:

The Two Delivery Models

Pub/Sub supports two ways to receive messages:

In this lab: You'll create a push subscription and see messages delivered automatically to a webhook, without needing to explicitly pull them.

Step 1: Open Google Cloud Console

Once your lab starts, use the "Open GCP Console" button from the lab portal to open the Google Cloud Console with your temporary lab project already selected.

Step 2: Navigate to Pub/Sub

1. In the Google Cloud Console, you can search for "Pub/Sub" in the search bar at the top
2. Or click the hamburger menu (three horizontal lines) and select Pub/Sub from the left navigation menu
3. Click on Topics

Step 3: Create a Topic

1. Click CREATE TOPIC at the top of the page
2. Enter a Topic ID starting with the required prefix: lab-topic-push-webhooks
3. IMPORTANT: Uncheck the "Add a default subscription" checkbox (we'll create a push subscription manually)
4. Leave all other settings at their defaults
5. Click CREATE

After creating the topic, you'll be viewing its details page. Click Topics in the left sidebar to see your new topic listed.

Step 4: Set Up Your Webhook Receiver

Before creating the push subscription, you need an HTTP endpoint that can receive messages. We'll use webhook.site, a free service that provides you with a unique URL and shows you all incoming HTTP requests in real-time.

1. Open a new browser tab and go to: https://webhook.site
2. The page will automatically generate a unique URL for you. It looks like: https://webhook.site/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
3. Copy this URL to your clipboard - you'll need it in the next step
4. Keep this tab open - you'll see messages arrive here in real-time when we publish them!

Note: This URL is temporary and unique to your session. Anyone with the URL can see the requests, but it expires after a period of inactivity.

Step 5: Create a Push Subscription

Now you'll create a push subscription that automatically delivers messages to your webhook.site URL.

1. In the Pub/Sub Topics list, click on your topic lab-topic-push-webhooks to view its details
2. Click CREATE SUBSCRIPTION
3. Configure the subscription:
   • Subscription ID: lab-subscription-push
   • Delivery type: Select Push (NOT Pull!)
   • Endpoint URL: Paste your webhook.site URL here (the one you copied in Step 4)
   • Leave all other settings at their defaults:
     • Acknowledgment deadline: 10 seconds
     • Message retention duration: 7 days
     • Retry policy: Retry immediately
     • Expiration period: 31 days
4. Click CREATE

What just happened? You configured Pub/Sub to automatically send HTTP POST requests to your webhook.site URL whenever a message is published to your topic. No pulling required!

Step 6: Publish Messages and Watch Real-Time Delivery

Now for the exciting part - publishing messages and seeing them delivered instantly!

1. Make sure your webhook.site tab is visible so you can watch messages arrive
2. In the Google Cloud Console, navigate back to Topics and click on your topic
3. Click the MESSAGES tab
4. Click PUBLISH MESSAGE
5. In the message body field, enter: Push subscription test message #1
6. Click PUBLISH
7. Immediately switch to your webhook.site tab - you should see a new HTTP POST request appear!
8. Click on the request in webhook.site to view details:
   • You'll see the message data (base64-encoded)
   • Message attributes and metadata
   • Pub/Sub-specific headers
9. Publish 2-3 more messages with different content and watch them arrive in real-time

Key observation: Notice how you didn't need to "pull" these messages. They were automatically delivered to your endpoint within seconds of being published. This is the power of push subscriptions!

Step 7: Understanding the Message Format

Look at the HTTP POST request body in webhook.site. Pub/Sub push subscriptions deliver messages in a specific JSON format:

{
  "message": {
    "data": "UHVzaCBzdWJzY3JpcHRpb24gdGVzdCBtZXNzYWdlICMx",
    "messageId": "1234567890",
    "publishTime": "2024-01-01T12:00:00.000Z",
    "attributes": {}
  },
  "subscription": "projects/your-project/subscriptions/lab-subscription-push"
}

• data: Your message content (base64-encoded)
• messageId: Unique identifier for the message
• publishTime: When the message was published
• attributes: Any custom attributes you added

Important for production: Your endpoint must respond with HTTP status code 200-299 within the acknowledgment deadline (10 seconds by default). If your endpoint returns an error or times out, Pub/Sub will retry delivery.

Step 8: Compare to Pull Subscriptions

Think back to the previous Pub/Sub Basics lab where you used pull subscriptions. Here are the key differences you've experienced:

Pull Subscription (Previous Lab):

• You had to manually click "PULL" to request messages
• Messages sat in the subscription queue until you pulled them
• You controlled when and how many messages to retrieve
• Good for batch processing and high-throughput scenarios

Push Subscription (This Lab):

• Messages were automatically delivered the moment you published them
• No manual pulling required - Pub/Sub sent HTTP requests to your endpoint
• Lower latency - messages arrived within seconds
• Good for real-time processing and event-driven architectures

Real-world example: Imagine you're building an order processing system. With push subscriptions, every new order could trigger an HTTP request to your application endpoint (whether it's a serverless function, containerized service, or any web application), which processes it immediately. With pull subscriptions, you might have a worker service that pulls batches of orders every minute to process them in bulk.

When to Use Push Subscriptions

• Serverless functions: Trigger Cloud Functions, Cloud Run services, or any serverless platform instantly when events occur
• Real-time notifications: Send user notifications immediately (e.g., order confirmations, alerts)
• Webhooks: Integrate with third-party services that expect webhook callbacks
• Event-driven microservices: Decouple services where each service has its own HTTP endpoint - this could be any application code running anywhere (containerized apps, VMs, on-premises servers, or managed services)
• Low-latency requirements: When you need to process events within seconds

Note: The subscriber endpoint can be any HTTP/HTTPS endpoint - serverless functions, containerized applications, traditional web servers, or even third-party services. As long as it can receive HTTP POST requests, it can be a push subscriber.

When to Use Pull Subscriptions

• Batch processing: Process messages in large batches for efficiency
• High throughput: When you need to control the rate of message consumption
• Complex processing: When message processing takes a long time
• Backpressure control: When your system needs to throttle incoming messages
• No HTTP endpoint: When your subscriber can't expose a public endpoint

Validation Fails

If your validation fails, check that:

• Your topic name starts with lab-topic-push (e.g., lab-topic-push-webhooks)
• Your subscription is named exactly lab-subscription-push (case-sensitive)
• The subscription delivery type is set to Push (not Pull)
• The subscription is attached to your topic
• You're using the same email address for validation that you used to start the lab

Messages Not Appearing in webhook.site

If messages aren't showing up in webhook.site:

• Verify you copied the correct webhook.site URL to the subscription endpoint
• Check that you're looking at the right webhook.site tab (the URL should match)
• Try refreshing the webhook.site page
• Verify the subscription shows "Push" as the delivery type (not "Pull")
• Check the subscription details to confirm the endpoint URL is correct
• Note: webhook.site URLs expire after inactivity - you may need to generate a new one

Can't Create Push Subscription

If you're having trouble creating the push subscription:

• Make sure you selected "Push" as the delivery type (it defaults to "Pull")
• The endpoint URL must be a valid HTTPS URL (webhook.site provides HTTPS)
• Don't include any authentication credentials in the basic setup (leave those fields blank)

Permission Denied Errors

If you get permission errors:

• Ensure you started the lab first and are signed into the correct Google account
• Check that you're working in the correct GCP project (shown in the top navigation bar)
• The lab session may have expired - start a new lab session