Migrate from Apache Cassandra to Bigtable

This document guides you through the process of migrating data from ApacheCassandra to Bigtable with minimal disruption. It describes how touse open-source tools, such as the Cassandra to Bigtable proxyadapter or the Cassandra to Bigtable client for Java, to performthe migration. Before you begin, make sure you're familiar withBigtable for Cassandra users.

Choose a migration approach

You can migrate from Apache Cassandra to Bigtable using one of thefollowing approaches:

• Cassandra to Bigtable proxy adapter lets youconnect Cassandra-based applications to Bigtable withoutchanging Cassandra drivers. This approach is ideal for applications thatrequire minimal code changes.
• Cassandra to Bigtable client for Java lets youintegrate directly with Bigtable and replace your Cassandradrivers. This approach is ideal for applications that require highperformance and flexibility.

Cassandra to Bigtable proxy adapter

The Cassandra to Bigtable proxy adapter lets you connectCassandra-based applications to Bigtable. The proxy adapterfunctions as a wire-compatible Cassandra interface, and it lets your applicationinteract with Bigtable using CQL. Using the proxy adapter doesn'trequire you to change Cassandra drivers, and configuration adjustments areminimal.

To set up and configure the proxy adapter, seeCassandra to Bigtable proxy adapter.

To learn which Cassandra versions support the proxy adapter, seeSupported Cassandra versions.

Limitations

The Cassandra to Bigtable proxy adapter provides limited supportfor certain data types, functions, queries, and clauses.

For more information, seeCassandra to Bigtable Proxy - Limitations.

Cassandra keyspace

A Cassandrakeyspace stores your tables and manages resources in a similar wayto a Bigtable instance. The Cassandra to Bigtableproxy adapter handles keyspace naming transparently, so that you canquery using the same keyspaces. However, you must create a newBigtableinstance to achieve logical grouping of yourtables. You must also configure Bigtable replication separately.

DDL support

The Cassandra to Bigtable proxy adapter supports Data DefinitionLanguage (DDL) operations. DDL operations let you create and manage tables directlythrough CQL commands. We recommend this approach for setting up your schemabecause it's similar to SQL but you don't need to define your schema inconfiguration files and then execute scripts to create tables.

The following examples show how the Cassandra to Bigtable proxyadapter supports DDL operations:

• To create a Cassandra table using CQL, run theCREATE TABLE command:

  CREATETABLEkeyspace.table(idbigint,nametext,ageint,PRIMARYKEY((id),name));

• To add a new column to the table, run theALTER TABLE command:

  ALTERTABLEkeyspace.tableADDemailtext;

• To delete a table, run theDROP TABLE command:

  DROPTABLEkeyspace.table;

For more information, seeDDL Support for Schema Creation (Recommended Method).

DML support

The Cassandra to Bigtable proxy adapter supports Data ManipulationLanguage (DML) operations such asINSERT,DELETE,UPDATE, andSELECT.

To run the raw DML queries, all values except numerics must have single quotes,as shown in the following examples:

• SELECT*FROMkeyspace.tableWHEREname='john doe';

• INSERTINTOkeyspace.table(id,name)VALUES(1,'john doe');

Achieve zero downtime migration

You can use the Cassandra to Bigtable proxy adapter with the open-sourceZero Downtime Migration (ZDM) proxy tooland theCassandra data migratortool to migrate data with minimal downtime.

The following diagram shows the steps for migrating from Cassandra toBigtable using the proxy adapter:

To migrate Cassandra to Bigtable, follow these steps:

1. Connect your Cassandra application to the ZDM proxy tool.
2. Enable dual writes to Cassandra and Bigtable.
3. Move data in bulk using theCassandra data migrator tool.
4. Validate your migration. Once validated, you can terminate the connection toCassandra and connect directly to Bigtable.

When using the proxy adapter with the ZDM proxy tool, the following migrationcapabilities are supported:

• Dual writes: maintain data availability during migration
• Asynchronous reads: scale and stress-test yourBigtable instance
• Automated data verification and reporting: ensure data integritythroughout the process
• Data mapping: map field and data types to meet your production standards

To practice migrating Cassandra to Bigtable, see theMigration from Cassandra to Bigtable with a Dual-Write Proxy codelab.

Cassandra to Bigtable client for Java

You can integrate directly with Bigtable and replace yourCassandra drivers, the Cassandra to Bigtable client for Javalibrary lets you integrate Cassandra-based Java applications withBigtable using CQL.

For instructions on building the library and including the dependency in application code, seeCassandra to Bigtable Client for Java.

The following example shows how to configure your application with the Cassandrato Bigtable client for Java:

importcom.datastax.oss.driver.api.core.CqlSession;importcom.datastax.oss.driver.api.core.cql.BoundStatement;importcom.datastax.oss.driver.api.core.cql.PreparedStatement;importcom.datastax.oss.driver.api.core.cql.ResultSet;importcom.datastax.oss.driver.api.core.cql.Row;importcom.google.bigtable.cassandra.BigtableCqlConfiguration;importcom.google.bigtable.cassandra.BigtableCqlSessionFactory;/** * Example using Bigtable CQLSession */publicclassExampleWithBigtableCqlSession{publicstaticvoidmain(String[]args){// Construct BigtableCqlConfigurationBigtableCqlConfigurationbigtableCqlConfiguration=BigtableCqlConfiguration.builder().setProjectId("example-project-id").setInstanceId("example-instance-id").setDefaultColumnFamily("example-column-family").setBigtableChannelPoolSize(4).build();// Create CqlSession with BigtableCqlConfigurationBigtableCqlSessionFactorybigtableCqlSessionFactory=newBigtableCqlSessionFactory(bigtableCqlConfiguration);// Create CqlSessiontry(CqlSessionsession=bigtableCqlSessionFactory.newSession()){// Create a tableStringcreateTableQuery="CREATE TABLE <KEYSPACE>.<TABLE_NAME> (<COLUMN> <TYPE> PRIMARY KEY);";session.execute(createTableQuery);// Prepare an insert statementPreparedStatementpreparedInsert=session.prepare("INSERT INTO <KEYSPACE>.<TABLE_NAME> (<COLUMN>) VALUES (?)"// replace with your keyspace, table and columns);// InsertBoundStatementboundInsert=preparedInsert.bind().setString("<COLUMN>","<VALUE>");session.execute(boundInsert);// Query for all entriesResultSetresultSet=session.execute("SELECT <COLUMN> FROM <KEYSPACE>.<TABLE_NAME>;");// Print resultsfor(Rowrow:resultSet){System.out.println(row);}}}}
ExampleWithBigtableCqlSession.java

Understand performance

Bigtable is designed for high throughput, low latency, and massivescalability. It can handle millions of requests per second on petabytes of data.When you migrate from Cassandra using either the Cassandra toBigtable client for Java or the Cassandra toBigtable proxy adapter, understand the following performanceimplications:

Client and proxy overhead

Both migration approaches introduce minimal performance overhead. They act as atranslation layer between the Cassandra Query Language (CQL) and theBigtable Data API, which is optimized for efficiency.

Performance with Cassandra collection types

If your Cassandra data model uses collection types such as maps, sets, or liststo implement dynamic schemas, then Bigtable can handle thesepatterns effectively. Both the proxy adapter and the client for Java map thesecollection operations to Bigtable's underlying data model, whichis well-suited for sparse and wide datasets.

Element-level operations within these collections are highlyefficient. This includes the following actions:

• To read or write a single value in a map.
• To add or remove an element from a set.
• To append or prepend an element to a list.

Bigtable optimizes these types of point operations on individualcollection elements, and their performance is identical to operations onstandard scalar columns.

Benchmark your workload

Bigtable performance can vary depending on your workload, schemadesign, data access patterns, and cluster configuration. To get accurateperformance metrics for your use case and ensure that Bigtablemeets your specific requirements, we recommend that you benchmark your Cassandraworkloads against Bigtable using one of the migration approaches.

For more information about Bigtable performance best practices,seeUnderstand performance.

Additional Cassandra open-source tools

Because of the wire compatibility of the Cassandra to Bigtable proxy adapterwith CQL lets you use additional tools in the Cassandra open-source ecosystem.These tools include the following:

• Cqlsh:The CQL shell lets you connect directly to Bigtable throughthe proxy adapter. You can use it for debugging and quick data lookups usingCQL.
• Cassandra Data Migrator (CDM):This Spark-based tool is suitable for migrating large volumes (up tobillions of rows) of historical data. The tool provides validation, diffreporting, and replay capabilities, and it's fully compatible with the proxyadapter.

What's next

• Schema design best practices