Pub/Sub: Publish and Subscribe to Messages

Step 0: Authentication

How this works: You need a Google account that uses the same email address as your GCP Study Hub account. When you sign into that Google account, the system will be able to grant you temporary access to a GCP project when you start the lab.

Click the button below to start your lab:

Step 1: Open Google Cloud Console

Once your lab starts, use the "Open GCP Console" button from the lab portal to open the Google Cloud Console with your temporary lab project already selected.

Step 2: Navigate to Pub/Sub

1. In the Google Cloud Console, you can search for "Pub/Sub" in the search bar at the top
2. Or click the hamburger menu (three horizontal lines) and select Pub/Sub from the left navigation menu
3. Click on Topics
4. If prompted, click Got it or Dismiss on any welcome messages

Step 3: Create a Topic

A topic is a named channel where publishers send messages. Think of it like a radio broadcast station - publishers send messages to the station, and anyone tuned in (subscribed) receives them.

1. Click CREATE TOPIC at the top of the page
2. Enter a Topic ID: lab-topic-notifications
3. IMPORTANT: You'll see a checkbox for "Add a default subscription" that is checked by default. Make sure to UNCHECK it. We'll create the subscription manually in the next step to better understand how subscriptions work.
4. Leave all other settings at their defaults:
   • Encryption: Google-managed encryption key (default)
   • Schema: None (schemas validate message structure, but we don't need that for this lab)
5. Click CREATE

You should see your new topic appear in the topics list. Topics can have multiple subscriptions, allowing different consumers to process the same messages independently.

Step 4: Create a Subscription

A subscription represents a stream of messages from a topic to a subscriber. Multiple subscriptions can exist on the same topic, and each subscription receives its own copy of every message.

1. After creating your topic, make sure you're viewing the topic details page for lab-topic-notifications (if you navigated away, click on the topic to view its details)
2. Scroll down and click CREATE SUBSCRIPTION
3. Configure the subscription (leave most settings at their defaults for this introductory lab):
   • Subscription ID: lab-subscription
   • Delivery type: Pull (default). This means your application explicitly requests messages, as opposed to Push where Pub/Sub sends HTTP requests to your endpoint.
   • Message retention duration: 7 days (default). Unacknowledged messages are kept for this long.
   • Acknowledgment deadline: 10 seconds (default is fine). This is how long Pub/Sub waits for acknowledgment before redelivering the message.
   • Transforms: Leave empty (default). Transforms let you manipulate and filter message data before delivery, but we don't need that for this lab.
   • Subscription filter: Leave empty (default). Filters let you receive only messages matching specific attribute criteria (e.g., attributes:k OR attributes.k1 = "v"). Useful for selective message processing.
   • Exactly once delivery: Leave unchecked (default). When enabled, guarantees messages won't be resent before acknowledgment deadline expires, eliminating duplicates. This increases latency and changes the acknowledgment deadline to 60 seconds minimum.
   • Message ordering: Leave unchecked (default). When enabled, messages with the same ordering key are delivered in the order they were published. This can increase latency and decrease availability.
   • Dead lettering: Leave unchecked (default). Dead lettering republishes undeliverable messages (after max delivery attempts) to a separate dead letter topic for later analysis.
   • Retry policy: Retry immediately (default). You can also choose "Retry after exponential backoff delay" which waits progressively longer between retries, useful for rate-limited systems.
   • Expiration period: 31 days (default). Inactive subscriptions are automatically deleted after this period to prevent accumulating unused resources.
4. Click CREATE

Your subscription is now ready to receive messages. Pub/Sub guarantees at-least-once delivery, meaning subscribers might occasionally see duplicate messages in rare failure scenarios.

Step 5: Publish Test Messages

Now you'll publish some test messages to your topic. In production systems, applications would publish messages programmatically, but the Console provides a convenient way to test.

1. Navigate back to Topics (click "Topics" in the left sidebar)
2. Click on your topic lab-topic-notifications
3. Click the MESSAGES tab
4. Click PUBLISH MESSAGE
5. In the message body field, enter: Hello from GCP Pub/Sub!
6. Optionally, you can add message attributes by clicking Add attribute:
   • Key: source
   • Value: manual-test
7. Click PUBLISH
8. Publish 2-3 more messages with different content (e.g., "Order #12345 processed", "User login event", etc.)

Each message is now queued in your subscription, waiting to be pulled and acknowledged.

Step 6: Pull and Acknowledge Messages

Now you'll pull messages from your subscription and learn how acknowledgment works. In production, your application code would do this automatically, but the Console lets us observe the behavior.

Part A: Pull without acknowledgment

1. Navigate to Subscriptions in the left sidebar
2. Click on lab-subscription
3. Click the MESSAGES tab
4. Make sure "Enable ack messages" is NOT checked (this is the default)
5. Click PULL
6. You should see the messages you published appear in the list
7. After about 10 seconds, you'll see "Deadline exceeded" appear next to the messages

What's happening? Without acknowledgment enabled, you're just viewing the messages. Since you didn't acknowledge them within the 10-second deadline, Pub/Sub will automatically make them available for redelivery. This ensures messages aren't lost if a subscriber crashes while processing them.

Part B: Pull with acknowledgment enabled

1. Now check the box next to "Enable ack messages" (this enables the ability to acknowledge)
2. Click PULL again
3. You should see the same messages reappear (they were redelivered because you didn't acknowledge them earlier)
4. Select one or more messages by checking the boxes next to them
5. Click ACK (acknowledge) to confirm you've successfully processed them
6. Notice the acknowledged messages disappear from the subscription - they've been permanently removed

What just happened? When you acknowledged the messages, you told Pub/Sub "I've successfully processed these messages." Pub/Sub then permanently removed them from the subscription queue. This is the core of Pub/Sub's at-least-once delivery guarantee: messages are redelivered until acknowledged, ensuring reliable message processing even in the face of subscriber failures.

Step 7: Understanding Message Redelivery

Let's verify you understand how acknowledgment works by observing message behavior:

1. Publish one more test message to your topic (go back to Topics → click your topic → MESSAGES tab → PUBLISH MESSAGE)
2. Return to your subscription (Subscriptions → lab-subscription → MESSAGES tab)
3. Pull the message with "Enable ack messages" checked
4. See the message appear, but don't acknowledge it yet
5. Wait about 15 seconds (longer than the 10-second acknowledgment deadline)
6. Click PULL again - the same message reappears (automatic redelivery!)
7. Now acknowledge the message - it disappears permanently

Key takeaway: Pub/Sub's reliability comes from this behavior. If your application crashes mid-processing, unacknowledged messages are automatically redelivered to another instance of your subscriber. This prevents message loss but means your code must be idempotent (safe to process the same message multiple times).

Validation Fails

If your validation fails, check that:

• Your topic name starts with lab-topic- (e.g., lab-topic-notifications)
• Your subscription is named exactly lab-subscription (case-sensitive)
• The subscription is attached to your topic (verify in the subscription details page under "Topic name")
• Both resources are in the correct project (check the project ID in the console header)
• You're using the same email address for validation that you used to start the lab

No Messages Appear When Pulling

If you don't see messages when pulling:

• Messages may have already been acknowledged - publish new messages and try again
• Verify you're pulling from the correct subscription (lab-subscription)
• Check that you published messages to the correct topic
• Verify the subscription is attached to the topic that received the messages

Permission Denied Errors

If you get permission errors:

• Ensure you started the lab first and are signed into the correct Google account
• Check that you're working in the correct GCP project (shown in the top navigation bar)
• The lab session may have expired - start a new lab session