You can route pipeline logs to a Cloud Pub/Sub sink to build an event-driven architecture. You can also sync pipeline logs into your observability tools. For more information about routing logs to a Cloud Pub/Sub sink, seeView logs routed to Pub/Sub.

This section describes how you can create a Pub/Sub topic, route (or sink) pipeline job logs to the Pub/Sub topic, and verify the logs published to the Pub/Sub topic. You can then subscribe downstream applications to the Pub/Sub topic to power event-driven architectures, or sync pipeline logs with your observability tools in near real-time.

Use the following instructions to set up a Pub/Sub sink and view the logs routed to it on the Google Cloud console:

1. Create a Pub/Sub topic. For more information, seeCreate a topic in the Cloud Pub/Sub documentation.

2. On theSubscriptions page of the Google Cloud console, create a pull subscription for the Pub/Sub topic. For more information, seeCreate pull subscription.

3. SelectLogging from the navigation menu, then clickGo to Log Router:
   

   Go to Log Router

4. In your project containing the logs you want to route, clickCreate Sink.

5. Enter a name and description for the sink, and then clickNext.

6. In theService list, selectCloud Pub/Sub topic.

7. In theDestination list, select the Pub/Sub topic that you created, and then clickNext.

8. UnderChoose logs to include in sink, specify the following inclusion filter:

   resource.type="aiplatform.googleapis.com/PipelineJob"

9. ClickCreate Sink.

10. On theSubscriptions page, click the subscription ID.

    Go to Subscriptions page

11. Click the subcription ID that you created.

12. On the subscription details page, click theMessages tab.

13. To pull and view the pipeline job logs, clickPull.