Datastream is a serverless and easy-to-use change data capture (CDC) and replication service that lets you synchronize data reliably, and with minimal latency.

Datastream provides seamless replication of data from operational databases into BigQuery. In addition, Datastream supports writing the change event stream into Cloud Storage, and offers streamlined integration with Dataflow templates to build custom workflows for loading data into a wide range of destinations, such as Cloud SQL and Spanner. You can also use Datastream to take advantage of the event stream directly from Cloud Storage to realize event-driven architectures. Datastream supports Oracle, MySQL, SQL Server, PostgreSQL (including AlloyDB for PostgreSQL), MongoDB and Salesforce sources.

Benefits of Datastream include:

• Seamless setup of ELT (Extract, Load, Transform) pipelines for low-latencydata replication to enable near real-time insights in BigQuery.
• Being serverless so there are no resources to provision or manage, and theservice scales up and down automatically, as needed, with minimal downtime.
• Easy-to-use setup and monitoring experiences that achieve super-fasttime-to-value.
• Integration across the best of Google Cloud data services'portfolio for data integration across Datastream,Dataflow, Pub/Sub, BigQuery, and more.
• Synchronizing and unifying data streams across heterogeneous databases andapplications.
• Security, with private connectivity options and the security you expect fromGoogle Cloud.
• Being accurate and reliable, with transparent status reporting and robustprocessing flexibility in the face of data and schema changes.
• Supporting multiple use cases, including analytics, database replication,and synchronization for migrations and hybrid-cloud configurations, and forbuilding event-driven architectures.

Use cases

The streaming capabilities of Datastream enable a variety of use cases:

• Replicating and synchronizing data across your organization with minimallatency

  You can synchronize data across heterogeneous databases and applicationsreliably, with low latency, and with minimal impact to the performance ofyour source. Unlock the power of data streams for analytics, databasereplication, cloud migration, and event-driven architectures across hybridenvironments.

• Scale up or down with a serverless architecture seamlessly

  Get up and running fast with a serverless and easy-to-use service thatscales seamlessly as your data volumes shift. Focus on deriving up-to-dateinsights from your data and responding to high-priority issues, instead ofmanaging infrastructure, performance tuning, or resource provisioning.

• Integrate with the Google Cloud data integration suite

  Connect data across your organization with the Google Cloud dataintegration suite of products. Integrate Datastream withDataflow job templates to read data from a Cloud Storage bucketand load it into a variety of destinations, such as BigQuery,Spanner, and Cloud SQL.

Experience elements

There are three main elements in Datastream:

• Private connectivity configurations enable Datastream tocommunicate with a data source over a private network (internally withinGoogle Cloud, or with external sources connected over VPN orInterconnect). This communication happens through a Virtual Private Cloud (VPC)peering connection.
• Connection profiles represent connectivity information to both a sourceand a destination. This information will be used by a stream.
• Streams use the information in the connection profiles to transfer CDCand backfill data from the source to the destination.

What's next

• Start replicating your datafrom a source database to BigQuery datasets.
• Learn more aboutkey concepts and features of Datastream.
• Find out how to createprivate connectivity configurations,connection profiles andstreams.