Streaming insert

Inserts simple rows into a table using the streaming API (insertAll).

Explore further

For detailed documentation that includes this code sample, see the following:

Code sample

C#

Before trying this sample, follow theC# setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryC# API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

usingGoogle.Cloud.BigQuery.V2;publicclassBigQueryTableInsertRows{publicvoidTableInsertRows(stringprojectId="your-project-id",stringdatasetId="your_dataset_id",stringtableId="your_table_id"){BigQueryClientclient=BigQueryClient.Create(projectId);BigQueryInsertRow[]rows=newBigQueryInsertRow[]{// The insert ID is optional, but can avoid duplicate data// when retrying inserts.newBigQueryInsertRow(insertId:"row1"){{"name","Washington"},{"post_abbr","WA"}},newBigQueryInsertRow(insertId:"row2"){{"name","Colorado"},{"post_abbr","CO"}}};client.InsertRows(datasetId,tableId,rows);}}

Go

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""cloud.google.com/go/bigquery")// Item represents a row item.typeItemstruct{NamestringAgeint}// Save implements the ValueSaver interface.// This example disables best-effort de-duplication, which allows for higher throughput.func(i*Item)Save()(map[string]bigquery.Value,string,error){returnmap[string]bigquery.Value{"full_name":i.Name,"age":i.Age,},bigquery.NoDedupeID,nil}// insertRows demonstrates inserting data into a table using the streaming insert mechanism.funcinsertRows(projectID,datasetID,tableIDstring)error{// projectID := "my-project-id"// datasetID := "mydataset"// tableID := "mytable"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %w",err)}deferclient.Close()inserter:=client.Dataset(datasetID).Table(tableID).Inserter()items:=[]*Item{// Item implements the ValueSaver interface.{Name:"Phred Phlyntstone",Age:32},{Name:"Wylma Phlyntstone",Age:29},}iferr:=inserter.Put(ctx,items);err!=nil{returnerr}returnnil}

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryError;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.InsertAllRequest;importcom.google.cloud.bigquery.InsertAllResponse;importcom.google.cloud.bigquery.TableId;importjava.util.HashMap;importjava.util.List;importjava.util.Map;// Sample to inserting rows into a table without running a load job.publicclassTableInsertRows{publicstaticvoidmain(String[]args){// TODO(developer): Replace these variables before running the sample.StringdatasetName="MY_DATASET_NAME";StringtableName="MY_TABLE_NAME";// Create a row to insertMap<String,Object>rowContent=newHashMap<>();rowContent.put("booleanField",true);rowContent.put("numericField","3.14");// TODO(developer): Replace the row id with a unique value for each row.StringrowId="ROW_ID";tableInsertRows(datasetName,tableName,rowId,rowContent);}publicstaticvoidtableInsertRows(StringdatasetName,StringtableName,StringrowId,Map<String,Object>rowContent){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();// Get tableTableIdtableId=TableId.of(datasetName,tableName);// Inserts rowContent into datasetName:tableId.InsertAllResponseresponse=bigquery.insertAll(InsertAllRequest.newBuilder(tableId)// More rows can be added in the same RPC by invoking .addRow() on the builder.// You can omit the unique row ids to disable de-duplication..addRow(rowId,rowContent).build());if(response.hasErrors()){// If any of the insertions failed, this lets you inspect the errorsfor(Map.Entry<Long,List<BigQueryError>>entry:response.getInsertErrors().entrySet()){System.out.println("Response error: \n"+entry.getValue());}}System.out.println("Rows successfully inserted into table");}catch(BigQueryExceptione){System.out.println("Insert operation not performed \n"+e.toString());}}}

Node.js

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Import the Google Cloud client libraryconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery();asyncfunctioninsertRowsAsStream(){// Inserts the JSON objects into my_dataset:my_table./**   * TODO(developer): Uncomment the following lines before running the sample.   */// const datasetId = 'my_dataset';// const tableId = 'my_table';constrows=[{name:'Tom',age:30},{name:'Jane',age:32},];// Insert data into a tableawaitbigquery.dataset(datasetId).table(tableId).insert(rows);console.log(`Inserted${rows.length} rows`);}

PHP

Before trying this sample, follow thePHP setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPHP API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

use Google\Cloud\BigQuery\BigQueryClient;/** * Stream data into bigquery * * @param string $projectId The project Id of your Google Cloud Project. * @param string $datasetId The BigQuery dataset ID. * @param string $tableId The BigQuery table ID. * @param string $data Json encoded data For eg, *    $data = json_encode([ *       "field1" => "value1", *       "field2" => "value2", *    ]); */function stream_row(    string $projectId,    string $datasetId,    string $tableId,    string $data): void {    // instantiate the bigquery table service    $bigQuery = new BigQueryClient([      'projectId' => $projectId,    ]);    $dataset = $bigQuery->dataset($datasetId);    $table = $dataset->table($tableId);    $data = json_decode($data, true);    $insertResponse = $table->insertRows([      ['data' => $data],      // additional rows can go here    ]);    if ($insertResponse->isSuccessful()) {        print('Data streamed into BigQuery successfully' . PHP_EOL);    } else {        foreach ($insertResponse->failedRows() as $row) {            foreach ($row['errors'] as $error) {                printf('%s: %s' . PHP_EOL, $error['reason'], $error['message']);            }        }    }}

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigquery# Construct a BigQuery client object.client=bigquery.Client()# TODO(developer): Set table_id to the ID of table to append to.# table_id = "your-project.your_dataset.your_table"rows_to_insert=[{"full_name":"Phred Phlyntstone","age":32},{"full_name":"Wylma Phlyntstone","age":29},]errors=client.insert_rows_json(table_id,rows_to_insert)# Make an API request.iferrors==[]:print("New rows have been added.")else:print("Encountered errors while inserting rows:{}".format(errors))

Ruby

Before trying this sample, follow theRuby setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryRuby API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

require"google/cloud/bigquery"deftable_insert_rowsdataset_id="your_dataset_id",table_id="your_table_id"bigquery=Google::Cloud::Bigquery.newdataset=bigquery.datasetdataset_idtable=dataset.tabletable_idrow_data=[{name:"Alice",value:5},{name:"Bob",value:10}]response=table.insertrow_dataifresponse.success?puts"Inserted rows successfully"elseputs"Failed to insert#{response.error_rows.count} rows"endend

What's next

To search and filter code samples for other Google Cloud products, see theGoogle Cloud sample browser.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.