Key Point: Understand what is Pub/Sub, why do businesses requirePub/Sub, and the advantages of Pub/Sub compared to similartechnologies. Also, learn about core Pub/Sub concepts thatinclude the terms topic, publisher, and subscriber.

Pub/Sub is an asynchronous and scalable messaging service that decouplesservices producing messages from services processing those messages.

Pub/Sub allows services to communicate asynchronously, withlatencies typically on the order of 100 milliseconds.

Pub/Sub is used for streaming analytics and data integrationpipelines to load and distribute data. It's equally effective as amessaging-oriented middleware for service integration or as a queue to parallelize tasks.

Pub/Sub lets you create systems of event producers and consumers,calledpublishers andsubscribers. Publishers communicate withsubscribers asynchronously by broadcasting events, rather than bysynchronous remote procedure calls (RPCs).

Publishers send events to the Pub/Sub service, without regard tohow or when these events are to be processed. Pub/Sub thendelivers events to all the services that react to them. In systems communicatingthrough RPCs, publishers must wait for subscribers to receive the data. However,the asynchronous integration in Pub/Sub increases the flexibilityand robustness of the overall system.

To get started with Pub/Sub, check out theQuickstart using Google Cloud console.For a more comprehensive introduction, seeBuilding a Pub/Sub messaging system.

Common use cases

• Ingesting user interaction and server events. To use user interaction events from end-user apps or server events from your system, you might forward them to Pub/Sub. You can then use a stream processing tool, such as Dataflow, which delivers the events to databases. Examples of such databases are BigQuery, Bigtable, and Cloud Storage. Pub/Sub lets you gather events from many clients simultaneously.
• Real-time event distribution. Events, raw or processed, may be made available to multiple applications across your team and organization for real- time processing. Pub/Sub supports an "enterprise event bus" and event-driven application design patterns. Pub/Sub lets you integrate with many systems that export events to Pub/Sub.
• Replicating data among databases. Pub/Sub is commonly used to distribute change events from databases. These events can be used to construct a view of the database state and state history in BigQuery and other data storage systems.
• Parallel processing and workflows. You can efficiently distribute many tasks among multiple workers by using Pub/Sub messages to communicate with the workers. Examples of such tasks are compressing text files, sending email notifications, evaluating AI models, and reformatting images.
• Enterprise event bus. You can create an enterprise-wide real-time data sharing bus, distributing business events, database updates, and analytics events across your organization.
• Data streaming from applications, services, or IoT devices. For example, a SaaS application can publish a real-time feed of events. Or, a residential sensor can stream data to Pub/Sub for use in other Google Cloud products through a data-processing pipeline.
• Refreshing distributed caches. For example, an application can publish invalidation events to update the IDs of objects that have changed.
• Load balancing for reliability. For example, instances of a service may be deployed on Compute Engine in multiple zones but subscribe to a common topic. When the service fails in any zone, the others can pick up the load automatically.

Comparing Pub/Sub to other messaging technologies

Pub/Sub combines the horizontal scalability ofApache Kafka andPulsar withfeatures found in messaging middleware such as Apache ActiveMQ andRabbitMQ. Examples of such features are dead-letter queues and filtering.

Note:Google Cloud Managed Service for Apache Kafkais available. If you're considering a migration from Kafka toPub/Sub, consultthis migration guide.

Another feature that Pub/Sub adopts from messaging middleware isper-message parallelism, rather than partition-based messaging.Pub/Sub "leases" individual messages to subscriber clients, thentracks whether a given message is successfully processed.

By contrast, other horizontally scalable messaging systemsuse partitions for horizontal scaling. This forces subscribersto process messages in each partition in order and limits the number of concurrentclients to the number of partitions. Per-message processingmaximizes the parallelism of subscriber applications, and helps ensurepublisher and subscriber independence.

Compare Service-to-service and service-to-client communication

Pub/Sub is intended for service-to-service communication ratherthan communication with end-user or IoT clients. Other patterns arebetter supported by other products:

• Client-server. To send messages between a mobile or web app and a service,use products that includeFirebase Realtime Database andFirebase Cloud Messaging.
• Asynchronous service calls. UseCloud Tasks.

You can use a combination of these services to build client -> services -> databasepatterns. For example, see the tutorialStreaming Pub/Sub messages over WebSockets.

Integrations

Pub/Sub has many integrations with other Google Cloud products to create a fullyfeatured messaging system:

• Stream processing and data integration. Supported byDataflow, including Dataflowtemplates andSQL, which allow processing anddata integration into BigQuery and data lakes on Cloud Storage. Dataflowtemplates for moving data from Pub/Sub toCloud Storage, BigQuery, and other products are available inthe Pub/Sub and Dataflow UIs in theGoogle Cloud console. Integration withApache Spark, particularly when managed withDataproc is also available. Visual composition of integration andprocessing pipelines running on Spark + Dataproc can be accomplished withData Fusion.
• Monitoring, Alerting and Logging. Supported by Monitoring andLogging products.
• Authentication and IAM. Pub/Sub relies on a standard OAuthauthentication used by other Google Cloud products and supports granular IAM,enabling access control for individual resources.
• APIs. Pub/Sub uses standardgRPC and REST service APItechnologies along withclient libraries for several languages.
• Triggers, notifications, and webhooks. Pub/Sub offers push-baseddelivery of messages as HTTP POST requests to webhooks. You can implement workflow automation usingCloud Functionsor other serverless products.
• Orchestration. Pub/Sub can be integrated into multistep serverlessWorkflows declaratively. Big data and analytic orchestration often done withCloud Composer, which supports Pub/Sub triggers.You can also integrate Pub/Sub withApplication Integration(Preview) which is anIntegration-Platform-as-a-Service (iPaaS) solution. Application Integration provides aPub/Sub triggerto trigger or start integrations.
• Integration Connectors.(Preview)Theseconnectors let you connect to various data sources.With connectors, both Google Cloud services and third-party business applications are exposedto your integrations through a transparent, standard interface. For Pub/Sub, you can create a Pub/Subconnectionfor use in your integrations.

Next steps

• Get started with the Pub/Subquickstart.
• Read thebasics of the Pub/Sub service.
• Learn how tobuild a Pub/Sub messaging system.
• Understand Pub/Subpricing.
• Understand quotas and limits forPub/Sub.
• Read the Pub/Subrelease notes.
• Explore data engineering with Google Cloud services on Qwiklabs.