Examples of de-identifying tabular data Stay organized with collections Save and categorize content based on your preferences.
Sensitive Data Protection can detect, classify, andde-identify sensitive data within structured data.When de-identifying content as a table, the structure and columns provideSensitive Data Protection with additional clues that may enable it to providebetter results for some use cases. For example, you can scan a single column fora certain data type instead of the entire table structure.
This topic provides examples of how to configure de-identification of sensitivedata within structured text. De-identification is enabled throughrecordtransformations. Thesetransformations are applied to values within tabular text data that areidentified as a specific infoType, or to an entire column of tabular data.
This topic also provides examples of tabular data transformations using thecryptographic hash method. The cryptographic transformation methods are uniquebecause of their requirement of a cryptographic key.
The JSON given in the following examples can be inserted into anyde-identification request inside the"deidentifyConfig"(DeidentifyConfig)attribute. Click the "APIs Explorer example" link to try out the example JSON inAPIsExplorer.
Transform a column without inspection
To transform a specific column in which the content is already known, youcan skip inspection and specify a transformation directly. The examplefollowing the tablebuckets the "HAPPINESS SCORE" columninto increments of 10.
| Input | Transformed table | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, seeSensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyUsingTableBucketing{publicstaticTableDeidentifyData(stringprojectId,TabletableToInspect=null){// Instantiate dlp client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToInspect==null){varrow1=newValue[]{newValue{StringValue="101"},newValue{StringValue="Charles Dickens"},newValue{StringValue="95"}};varrow2=newValue[]{newValue{StringValue="22"},newValue{StringValue="Jane Austin"},newValue{StringValue="21"}};varrow3=newValue[]{newValue{StringValue="55"},newValue{StringValue="Mark Twain"},newValue{StringValue="75"}};tableToInspect=newTable{Headers={newFieldId{Name="AGE"},newFieldId{Name="PATIENT"},newFieldId{Name="HAPPINESS SCORE"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Construct the table content item.varcontentItem=newContentItem{Table=tableToInspect};// Specify how the content should be de-identified.varfixedSizeBucketingConfig=newFixedSizeBucketingConfig{BucketSize=10,LowerBound=newValue{IntegerValue=0},UpperBound=newValue{IntegerValue=100},};// Specify the fields to be encrypted.varfields=newFieldId[]{newFieldId{Name="HAPPINESS SCORE"}};// Associate the encryption with the specified field.varfieldTransformation=newFieldTransformation{PrimitiveTransformation=newPrimitiveTransformation{FixedSizeBucketingConfig=fixedSizeBucketingConfig},Fields={fields}};// Construct the deidentify config.vardeidentifyConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={fieldTransformation}}};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifyConfig,Item=contentItem,};// Call the API.varresponse=dlp.DeidentifyContent(request);// Inspect the response.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deIdentifyTableBucketing de-identifies data using table bucketingfuncdeIdentifyTableBucketing(wio.Writer,projectIDstring)error{// projectId := "your-project-id"// table := "your-table-value"row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"22"}},{Type:&dlppb.Value_StringValue{StringValue:"Jane Austen"}},{Type:&dlppb.Value_StringValue{StringValue:"21"}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"55"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain"}},{Type:&dlppb.Value_StringValue{StringValue:"75"}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"101"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens"}},{Type:&dlppb.Value_StringValue{StringValue:"95"}},},}table:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"AGE"},{Name:"PATIENT"},{Name:"HAPPINESS SCORE"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:table,},}// Specify how the content should be de-identified.fixedSizeBucketingConfig:=&dlppb.FixedSizeBucketingConfig{BucketSize:10,LowerBound:&dlppb.Value{Type:&dlppb.Value_IntegerValue{IntegerValue:0,},},UpperBound:&dlppb.Value{Type:&dlppb.Value_IntegerValue{IntegerValue:100,},},}primitiveTransformation:=&dlppb.PrimitiveTransformation_FixedSizeBucketingConfig{FixedSizeBucketingConfig:fixedSizeBucketingConfig,}// Specify field to be encrypted.fieldId:=&dlppb.FieldId{Name:"HAPPINESS SCORE",}// Associate the encryption with the specified field.fieldTransformation:=&dlppb.FieldTransformation{Transformation:&dlppb.FieldTransformation_PrimitiveTransformation{PrimitiveTransformation:&dlppb.PrimitiveTransformation{Transformation:primitiveTransformation,},},Fields:[]*dlppb.FieldId{fieldId,},}recordTransformations:=&dlppb.RecordTransformations{FieldTransformations:[]*dlppb.FieldTransformation{fieldTransformation,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},},Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.FixedSizeBucketingConfig;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;publicclassDeIdentifyTableBucketing{publicstaticvoidmain(String[]args)throwsIOException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("AGE").build()).addHeaders(FieldId.newBuilder().setName("PATIENT").build()).addHeaders(FieldId.newBuilder().setName("HAPPINESS SCORE").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("101").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens").build()).addValues(Value.newBuilder().setStringValue("95").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("22").build()).addValues(Value.newBuilder().setStringValue("Jane Austen").build()).addValues(Value.newBuilder().setStringValue("21").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("55").build()).addValues(Value.newBuilder().setStringValue("Mark Twain").build()).addValues(Value.newBuilder().setStringValue("75").build()).build()).build();deIdentifyTableBucketing(projectId,tableToDeIdentify);}publicstaticTabledeIdentifyTableBucketing(StringprojectId,TabletableToDeIdentify)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to de-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify how the content should be de-identified.FixedSizeBucketingConfigfixedSizeBucketingConfig=FixedSizeBucketingConfig.newBuilder().setBucketSize(10).setLowerBound(Value.newBuilder().setIntegerValue(0).build()).setUpperBound(Value.newBuilder().setIntegerValue(100).build()).build();PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setFixedSizeBucketingConfig(fixedSizeBucketingConfig).build();// Specify field to be encrypted.FieldIdfieldId=FieldId.newBuilder().setName("HAPPINESS SCORE").build();// Associate the encryption with the specified field.FieldTransformationfieldTransformation=FieldTransformation.newBuilder().setPrimitiveTransformation(primitiveTransformation).addFields(fieldId).build();RecordTransformationstransformations=RecordTransformations.newBuilder().addFieldTransformations(fieldTransformation).build();DeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after de-identification: "+response.getItem().getTable());returnresponse.getItem().getTable();}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Initialize google DLP Clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Construct the tabular dataconsttablularData={headers:[{name:'AGE'},{name:'PATIENT'},{name:'HAPPINESS SCORE'}],rows:[{values:[{integerValue:101},{stringValue:'Charles Dickens'},{integerValue:95},],},{values:[{integerValue:22},{stringValue:'Jane Austen'},{integerValue:21},],},{values:[{integerValue:55},{stringValue:'Mark Twain'},{integerValue:75},],},],};asyncfunctiondeIdentifyTableBucketing(){// Specify field to be de-identified.consttargetColumn={name:'HAPPINESS SCORE'};// Specify how the content should be de-identified.constbucketingConfig={bucketSize:10,lowerBound:{integerValue:0,},upperBound:{integerValue:100,},};constprimitiveTransformation={fixedSizeBucketingConfig:bucketingConfig,};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,item:{table:tablularData,},deidentifyConfig:{recordTransformations:{fieldTransformations:[{fields:[targetColumn],primitiveTransformation,},],},},};// Send the request and receive response from the serviceconst[response]=awaitdlp.deidentifyContent(request);// Print the results.console.log(`Table after de-identification:${JSON.stringify(response.item.table)}`);}deIdentifyTableBucketing();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\FixedSizeBucketingConfig;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * De-identify data using table bucketing * Transform a column without inspection. To transform a column in which the content is * already known, you can skip inspection and specify a transformation directly. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify * @param string $outputCsvFile The oupt file path to save deidentify content * */function deidentify_table_bucketing( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table2.csv', string $outputCsvFile = './test/data/deidentify_table_bucketing_output.csv'): void { // Instantiate a client. $dlp = new DlpServiceClient(); // Read a CSV file $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify what content you want the service to de-identify. $contentItem = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify how the content should be de-identified. $fixedSizeBucketingConfig = (new FixedSizeBucketingConfig()) ->setBucketSize(10) ->setLowerBound((new Value()) ->setIntegerValue(10)) ->setUpperBound((new Value()) ->setIntegerValue(100)); $primitiveTransformation = (new PrimitiveTransformation()) ->setFixedSizeBucketingConfig($fixedSizeBucketingConfig); // Specify the field to to apply bucketing transform on $fieldId = (new FieldId()) ->setName('HAPPINESS_SCORE'); // Associate the encryption with the specified field. $fieldTransformation = (new FieldTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setFields([$fieldId]); $recordTransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation]); // Create the deidentification configuration object $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordTransformations); $parent = "projects/$callingProjectId/locations/global"; // Run request $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setDeidentifyConfig($deidentifyConfig) ->setItem($contentItem); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print results $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf($outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpfromgoogle.cloud.dlp_v2importtypesdefdeidentify_table_bucketing(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],deid_content_list:List[str],bucket_size:int,bucketing_lower_bound:int,bucketing_upper_bound:int,)->types.dlp.Table:"""Uses the Data Loss Prevention API to de-identify sensitive data in a table by replacing them with fixed size bucket ranges. Args: project: The Google Cloud project id to use as a parent resource. table_data: Dictionary representing table data. deid_content_list: A list of fields in table to de-identify. bucket_size: Size of each bucket for fixed sized bucketing (except for minimum and maximum buckets). So if ``bucketing_lower_bound`` = 10, ``bucketing_upper_bound`` = 89, and ``bucket_size`` = 10, then the following buckets would be used: -10, 10-20, 20-30, 30-40, 40-50, 50-60, 60-70, 70-80, 80-89, 89+. bucketing_lower_bound: Lower bound value of buckets. bucketing_upper_bound: Upper bound value of buckets. Returns: De-identified table is returned; the response from the API is also printed to the terminal. Example: >> $ python deidentify_table_bucketing.py \ '{"header": ["email", "phone number", "age"], "rows": [["robertfrost@example.com", "4232342345", "35"], ["johndoe@example.com", "4253458383", "68"]]}' \ ["age"] 10 0 100 >> '{"header": ["email", "phone number", "age"], "rows": [["robertfrost@example.com", "4232342345", "30:40"], ["johndoe@example.com", "4253458383", "60:70"]]}' """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the `item`.item={"table":table}# Construct fixed sized bucketing configurationfixed_size_bucketing_config={"bucket_size":bucket_size,"lower_bound":{"integer_value":bucketing_lower_bound},"upper_bound":{"integer_value":bucketing_upper_bound},}# Specify fields to be de-identifieddeid_content_list=[{"name":_i}for_iindeid_content_list]# Construct Deidentify Configdeidentify_config={"record_transformations":{"field_transformations":[{"fields":deid_content_list,"primitive_transformation":{"fixed_size_bucketing_config":fixed_size_bucketing_config},}]}}# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"item":item})# Print the results.print(f"Table after de-identification:{response.item.table}")# Return the response.returnresponse.item.table"deidentifyConfig":{ "recordTransformations":{ "fieldTransformations":[ { "fields":[ { "name":"HAPPINESS SCORE" } ], "primitiveTransformation":{ "fixedSizeBucketingConfig":{ "bucketSize":10, "lowerBound":{ "integerValue":"0" }, "upperBound":{ "integerValue":"100" } } } } ] }}Transform a column based on the value of another column
You can transform a column based on the value of another. This example redacts"HAPPINESS SCORE" for all patients over 89.
| Input | Transformed table | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, seeSensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyTableUsingMaskingAndLogic{publicstaticTableDeidentifyTable(stringprojectId,TabletableToInspect=null){// Instantiate the client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToInspect==null){varrow1=newValue[]{newValue{StringValue="101"},newValue{StringValue="Charles Dickens"},newValue{StringValue="95"},newValue{StringValue="Charles Dickens name was a curse invented by Shakespeare."}};varrow2=newValue[]{newValue{StringValue="22"},newValue{StringValue="Jane Austin"},newValue{StringValue="21"},newValue{StringValue="There are 14 kisses in Jane Austen's novels."}};varrow3=newValue[]{newValue{StringValue="55"},newValue{StringValue="Mark Twain"},newValue{StringValue="75"},newValue{StringValue="Mark Twain loved cats."}};tableToInspect=newTable{Headers={newFieldId{Name="AGE"},newFieldId{Name="PATIENT"},newFieldId{Name="HAPPINESS SCORE"},newFieldId{Name="FACTOID"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Construct the table content item.varcontentItem=newContentItem{Table=tableToInspect};//Specify how the content should be de-identified.varprimitiveTransformation=newPrimitiveTransformation{CharacterMaskConfig=newCharacterMaskConfig{MaskingCharacter="*"}};// Specify the fields to be de-identified.varfields=newFieldId[]{newFieldId{Name="HAPPINESS SCORE"}};// Specify when the above fields should be de-identified using condition.varcondition=newRecordCondition.Types.Conditions{Conditions_={newRecordCondition.Types.Condition{Field=newFieldId{Name="AGE"},Operator=RelationalOperator.GreaterThan,Value=newValue{IntegerValue=89}}}};// Apply the condition to recordsvarrecordCondition=newRecordCondition{Expressions=newRecordCondition.Types.Expressions{Conditions=condition}};// Associate the de-identification and conditions with the specified fields.vardeidentifiedConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={newFieldTransformation{PrimitiveTransformation=primitiveTransformation,Fields={fields},Condition=recordCondition}}}};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifiedConfig,Item=contentItem};// Call the API.varresponse=dlp.DeidentifyContent(request);// Inspect the response.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deidentifyTableMaskingCondition de-identifies the table data using masking// and conditional logicfuncdeidentifyTableMaskingCondition(wio.Writer,projectIDstring)error{// projectId := "your-project-id"row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"22"}},{Type:&dlppb.Value_StringValue{StringValue:"Jane Austen"}},{Type:&dlppb.Value_StringValue{StringValue:"21"}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"55"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain"}},{Type:&dlppb.Value_StringValue{StringValue:"75"}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"101"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens"}},{Type:&dlppb.Value_StringValue{StringValue:"95"}},},}table:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"AGE"},{Name:"PATIENT"},{Name:"HAPPINESS SCORE"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:table,},}// Specify how the content should be de-identified.charMaskConfig:=&dlppb.CharacterMaskConfig{MaskingCharacter:"*",}primitiveTransformation:=&dlppb.PrimitiveTransformation_CharacterMaskConfig{CharacterMaskConfig:charMaskConfig,}// Specify field to be de-identified.fieldId:=&dlppb.FieldId{Name:"HAPPINESS SCORE",}// Apply the condition to the records present in table.recordCondition:=&dlppb.RecordCondition_Condition{Field:fieldId,Operator:dlppb.RelationalOperator_GREATER_THAN,Value:&dlppb.Value{Type:&dlppb.Value_IntegerValue{IntegerValue:89,},},}expression:=&dlppb.RecordCondition_Expressions{Type:&dlppb.RecordCondition_Expressions_Conditions{Conditions:&dlppb.RecordCondition_Conditions{Conditions:[]*dlppb.RecordCondition_Condition{recordCondition,},},},}// Specify when the above field should be de-identified.condition:=&dlppb.RecordCondition{// Apply the condition to recordsExpressions:expression,}// Associate the de-identification and conditions with the specified field.fieldTransformation:=&dlppb.FieldTransformation{Fields:[]*dlppb.FieldId{fieldId,},Transformation:&dlppb.FieldTransformation_PrimitiveTransformation{PrimitiveTransformation:&dlppb.PrimitiveTransformation{Transformation:primitiveTransformation,},},Condition:condition,}recordTransformations:=&dlppb.RecordTransformations{FieldTransformations:[]*dlppb.FieldTransformation{fieldTransformation,},}// Combine configurations into a request for the service.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},},Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.CharacterMaskConfig;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordCondition;importcom.google.privacy.dlp.v2.RecordCondition.Condition;importcom.google.privacy.dlp.v2.RecordCondition.Conditions;importcom.google.privacy.dlp.v2.RecordCondition.Expressions;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.RelationalOperator;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;publicclassDeIdentifyTableConditionMasking{publicstaticvoidmain(String[]args)throwsIOException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("AGE").build()).addHeaders(FieldId.newBuilder().setName("PATIENT").build()).addHeaders(FieldId.newBuilder().setName("HAPPINESS SCORE").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("101").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens").build()).addValues(Value.newBuilder().setStringValue("95").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("22").build()).addValues(Value.newBuilder().setStringValue("Jane Austen").build()).addValues(Value.newBuilder().setStringValue("21").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("55").build()).addValues(Value.newBuilder().setStringValue("Mark Twain").build()).addValues(Value.newBuilder().setStringValue("75").build()).build()).build();deIdentifyTableConditionMasking(projectId,tableToDeIdentify);}publicstaticTabledeIdentifyTableConditionMasking(StringprojectId,TabletableToDeIdentify)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to de-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify how the content should be de-identified.CharacterMaskConfigcharacterMaskConfig=CharacterMaskConfig.newBuilder().setMaskingCharacter("*").build();PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setCharacterMaskConfig(characterMaskConfig).build();// Specify field to be de-identified.FieldIdfieldId=FieldId.newBuilder().setName("HAPPINESS SCORE").build();// Specify when the above field should be de-identified.Conditioncondition=Condition.newBuilder().setField(FieldId.newBuilder().setName("AGE").build()).setOperator(RelationalOperator.GREATER_THAN).setValue(Value.newBuilder().setIntegerValue(89).build()).build();// Apply the condition to recordsRecordConditionrecordCondition=RecordCondition.newBuilder().setExpressions(Expressions.newBuilder().setConditions(Conditions.newBuilder().addConditions(condition).build()).build()).build();// Associate the de-identification and conditions with the specified field.FieldTransformationfieldTransformation=FieldTransformation.newBuilder().setPrimitiveTransformation(primitiveTransformation).addFields(fieldId).setCondition(recordCondition).build();RecordTransformationstransformations=RecordTransformations.newBuilder().addFieldTransformations(fieldTransformation).build();DeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after de-identification: "+response.getItem().getTable());returnresponse.getItem().getTable();}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Initialize google DLP Clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Construct the tabular dataconsttablularData={headers:[{name:'AGE'},{name:'PATIENT'},{name:'HAPPINESS SCORE'}],rows:[{values:[{integerValue:101},{stringValue:'Charles Dickens'},{integerValue:95},],},{values:[{integerValue:22},{stringValue:'Jane Austen'},{integerValue:21},],},{values:[{integerValue:55},{stringValue:'Mark Twain'},{integerValue:75},],},],};asyncfunctiondeIdentifyTableConditionalCharMasking(){// Column that needs to be transformedconstfieldIds=[{name:'HAPPINESS SCORE'}];// Construct PrimitiveTransformation configurationconstprimitiveTransformation={characterMaskConfig:{maskingCharacter:'*',},};// Construct conditionconstcondition={expressions:{conditions:{conditions:[{field:{name:'AGE'},operator:'GREATER_THAN',value:{integerValue:89},},],},},};// Construct RecordTransformations configurationsconstrecordTransformations={fieldTransformations:[{primitiveTransformation,fields:fieldIds,condition,},],};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,item:{table:tablularData,},deidentifyConfig:{recordTransformations,},};// Send the request and receive response from the serviceconst[response]=awaitdlp.deidentifyContent(request);// Print the resultsconsole.log(`Table after de-identification:${JSON.stringify(response.item.table)}`);}deIdentifyTableConditionalCharMasking();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\CharacterMaskConfig;use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordCondition;use Google\Cloud\Dlp\V2\RecordCondition\Condition;use Google\Cloud\Dlp\V2\RecordCondition\Conditions;use Google\Cloud\Dlp\V2\RecordCondition\Expressions;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\RelationalOperator;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data using masking and conditional logic. * Transform a column based on the value of another column. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify * @param string $outputCsvFile The oupt file path to save deidentify content */function deidentify_table_condition_masking( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table2.csv', string $outputCsvFile = './test/data/deidentify_table_condition_masking_output.csv'): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify what content you want the service to de-identify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify how the content should be de-identified. $characterMaskConfig = (new CharacterMaskConfig()) ->setMaskingCharacter('*'); $primitiveTransformation = (new PrimitiveTransformation()) ->setCharacterMaskConfig($characterMaskConfig); // Specify field to be de-identified. $fieldId = (new FieldId()) ->setName('HAPPINESS_SCORE'); // Specify when the above fields should be de-identified. $condition = (new Condition()) ->setField((new FieldId()) ->setName('AGE')) ->setOperator(RelationalOperator::GREATER_THAN) ->setValue((new Value()) ->setIntegerValue(89)); // Apply the condition to records $recordCondition = (new RecordCondition()) ->setExpressions((new Expressions()) ->setConditions((new Conditions()) ->setConditions([$condition]) ) ); // Associate the de-identification and conditions with the specified fields. $fieldTransformation = (new FieldTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setFields([$fieldId]) ->setCondition($recordCondition); $recordtransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation]); $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Run request $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print results $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf('After de-identify the table data (Output File Location): %s', $outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpfromgoogle.cloud.dlp_v2importtypesdefdeidentify_table_condition_masking(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],deid_content_list:List[str],condition_field:str=None,condition_operator:str=None,condition_value:int=None,masking_character:str=None,)->types.dlp.Table:""" Uses the Data Loss Prevention API to de-identify sensitive data in a table by masking them based on a condition. Args: project: The Google Cloud project id to use as a parent resource. table_data: Json string representing table data. deid_content_list: A list of fields in table to de-identify. condition_field: A table Field within the record this condition is evaluated against. condition_operator: Operator used to compare the field or infoType to the value. One of: RELATIONAL_OPERATOR_UNSPECIFIED, EQUAL_TO, NOT_EQUAL_TO, GREATER_THAN, LESS_THAN, GREATER_THAN_OR_EQUALS, LESS_THAN_OR_EQUALS, EXISTS. condition_value: Value to compare against. [Mandatory, except for ``EXISTS`` tests.]. masking_character: The character to mask matching sensitive data with. Returns: De-identified table is returned; the response from the API is also printed to the terminal. Example: >> $ python deidentify_table_condition_masking.py \ '{"header": ["email", "phone number", "age", "happiness_score"], "rows": [["robertfrost@example.com", "4232342345", "35", "21"], ["johndoe@example.com", "4253458383", "64", "34"]]}' \ ["happiness_score"] "age" "GREATER_THAN" 50 >> '{"header": ["email", "phone number", "age", "happiness_score"], "rows": [["robertfrost@example.com", "4232342345", "35", "21"], ["johndoe@example.com", "4253458383", "64", "**"]]}' """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the `item`item={"table":table}# Specify fields to be de-identifieddeid_content_list=[{"name":_i}for_iindeid_content_list]# Construct condition listcondition=[{"field":{"name":condition_field},"operator":condition_operator,"value":{"integer_value":condition_value},}]# Construct deidentify configuration dictionarydeidentify_config={"record_transformations":{"field_transformations":[{"primitive_transformation":{"character_mask_config":{"masking_character":masking_character}},"fields":deid_content_list,"condition":{"expressions":{"conditions":{"conditions":condition}}},}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"item":item})# Print the resultprint(f"Table after de-identification:{response.item.table}")# Return the responsereturnresponse.item.table"deidentifyConfig":{ "recordTransformations":{ "fieldTransformations":[ { "fields":[ { "name":"HAPPINESS SCORE" } ], "primitiveTransformation":{ "characterMaskConfig":{ "maskingCharacter":"*" } }, "condition":{ "expressions":{ "conditions":{ "conditions":[ { "field":{ "name":"AGE" }, "operator":"GREATER_THAN", "value":{ "integerValue":"89" } } ] } } } } ] }}Transform findings found in columns
You can transform findings that make up either only part of a cell'scontent or all of it. In this example, all instances ofPERSON_NAME areanonymized.
| Input | Transformed table | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, seeSensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingSystem.Collections.Generic;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyTableWithInfoTypes{publicstaticTableDeidentifyTable(stringprojectId,TabletableToInspect=null,IEnumerable<InfoType>infoTypes=null){// Instantiate a client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToInspect==null){varrow1=newValue[]{newValue{StringValue="101"},newValue{StringValue="Charles Dickens"},newValue{StringValue="95"},newValue{StringValue="Charles Dickens name was a curse invented by Shakespeare."}};varrow2=newValue[]{newValue{StringValue="22"},newValue{StringValue="Jane Austin"},newValue{StringValue="21"},newValue{StringValue="There are 14 kisses in Jane Austen's novels."}};varrow3=newValue[]{newValue{StringValue="55"},newValue{StringValue="Mark Twain"},newValue{StringValue="75"},newValue{StringValue="Mark Twain loved cats."}};tableToInspect=newTable{Headers={newFieldId{Name="AGE"},newFieldId{Name="PATIENT"},newFieldId{Name="HAPPINESS SCORE"},newFieldId{Name="FACTOID"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Construct the table content item.varcontentItem=newContentItem{Table=tableToInspect};// Construct Replace With InfoTypes config to replace the match.varreplaceInfoTypesConfig=newReplaceWithInfoTypeConfig();// Construct Fields to be de-identified.varfieldIds=newFieldId[]{newFieldId{Name="PATIENT"},newFieldId{Name="FACTOID"}};// Construct InfoType Transformation.varinfoTypeTransformations=newInfoTypeTransformations{Transformations={newInfoTypeTransformations.Types.InfoTypeTransformation{PrimitiveTransformation=newPrimitiveTransformation{ReplaceWithInfoTypeConfig=replaceInfoTypesConfig},InfoTypes={infoTypes??newInfoType[]{newInfoType{Name="PERSON_NAME"}}}}}};// Construct the de-identify config using replace config.vardeidentifyConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={newFieldTransformation{InfoTypeTransformations=infoTypeTransformations,Fields={fieldIds}}}}};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifyConfig,Item=contentItem};// Call the API.varresponse=dlp.DeidentifyContent(request);// Inspect the response.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deidentifyTableInfotypes de-identifies table data with info typesfuncdeidentifyTableInfotypes(wio.Writer,projectIDstring)error{// projectId := "your-project-id"row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"22"}},{Type:&dlppb.Value_StringValue{StringValue:"Jane Austen"}},{Type:&dlppb.Value_StringValue{StringValue:"21"}},{Type:&dlppb.Value_StringValue{StringValue:"There are 14 kisses in Jane Austen's novels."}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"55"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain"}},{Type:&dlppb.Value_StringValue{StringValue:"75"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain loved cats."}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"101"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens"}},{Type:&dlppb.Value_StringValue{StringValue:"95"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens name was a curse invented by Shakespeare."}},},}table:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"AGE"},{Name:"PATIENT"},{Name:"HAPPINESS SCORE"},{Name:"FACTOID"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewRESTClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:table,},}// Specify how the content should be de-identified.// Select type of info to be replaced.infoTypes:=[]*dlppb.InfoType{{Name:"PERSON_NAME"},}// Specify that findings should be replaced with corresponding info type name.replaceWithInfoTypeConfig:=&dlppb.ReplaceWithInfoTypeConfig{}primitiveTransformation:=&dlppb.PrimitiveTransformation{Transformation:&dlppb.PrimitiveTransformation_ReplaceWithInfoTypeConfig{ReplaceWithInfoTypeConfig:replaceWithInfoTypeConfig,},}// Associate info type with the replacement strategyinfoTypeTransformations:=&dlppb.InfoTypeTransformations{Transformations:[]*dlppb.InfoTypeTransformations_InfoTypeTransformation{{InfoTypes:infoTypes,PrimitiveTransformation:primitiveTransformation,},},}// Specify fields to be de-identified.fields:=[]*dlppb.FieldId{{Name:"PATIENT"},{Name:"FACTOID"},}// Associate the de-identification and conditions with the specified field.fieldTransformation:=&dlppb.FieldTransformation{Fields:fields,Transformation:&dlppb.FieldTransformation_InfoTypeTransformations{InfoTypeTransformations:infoTypeTransformations,},}recordTransformations:=&dlppb.RecordTransformations{FieldTransformations:[]*dlppb.FieldTransformation{fieldTransformation,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},},Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.InfoType;importcom.google.privacy.dlp.v2.InfoTypeTransformations;importcom.google.privacy.dlp.v2.InfoTypeTransformations.InfoTypeTransformation;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.ReplaceWithInfoTypeConfig;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;importjava.util.List;importjava.util.stream.Collectors;importjava.util.stream.Stream;publicclassDeIdentifyTableInfoTypes{publicstaticvoidmain(String[]args)throwsIOException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("AGE").build()).addHeaders(FieldId.newBuilder().setName("PATIENT").build()).addHeaders(FieldId.newBuilder().setName("HAPPINESS SCORE").build()).addHeaders(FieldId.newBuilder().setName("FACTOID").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("101").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens").build()).addValues(Value.newBuilder().setStringValue("95").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens name was a curse invented by Shakespeare.").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("22").build()).addValues(Value.newBuilder().setStringValue("Jane Austen").build()).addValues(Value.newBuilder().setStringValue("21").build()).addValues(Value.newBuilder().setStringValue("There are 14 kisses in Jane Austen's novels.").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("55").build()).addValues(Value.newBuilder().setStringValue("Mark Twain").build()).addValues(Value.newBuilder().setStringValue("75").build()).addValues(Value.newBuilder().setStringValue("Mark Twain loved cats.").build()).build()).build();deIdentifyTableInfoTypes(projectId,tableToDeIdentify);}publicstaticTabledeIdentifyTableInfoTypes(StringprojectId,TabletableToDeIdentify)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to de-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify how the content should be de-identified.// Select type of info to be replaced.InfoTypeinfoType=InfoType.newBuilder().setName("PERSON_NAME").build();// Specify that findings should be replaced with corresponding info type name.ReplaceWithInfoTypeConfigreplaceWithInfoTypeConfig=ReplaceWithInfoTypeConfig.getDefaultInstance();PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setReplaceWithInfoTypeConfig(replaceWithInfoTypeConfig).build();// Associate info type with the replacement strategyInfoTypeTransformationinfoTypeTransformation=InfoTypeTransformation.newBuilder().addInfoTypes(infoType).setPrimitiveTransformation(primitiveTransformation).build();InfoTypeTransformationsinfoTypeTransformations=InfoTypeTransformations.newBuilder().addTransformations(infoTypeTransformation).build();// Specify fields to be de-identified.List<FieldId>fieldIds=Stream.of("PATIENT","FACTOID").map(id->FieldId.newBuilder().setName(id).build()).collect(Collectors.toList());// Associate the de-identification and conditions with the specified field.FieldTransformationfieldTransformation=FieldTransformation.newBuilder().setInfoTypeTransformations(infoTypeTransformations).addAllFields(fieldIds).build();RecordTransformationstransformations=RecordTransformations.newBuilder().addFieldTransformations(fieldTransformation).build();DeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after de-identification: "+response.getItem().getTable());returnresponse.getItem().getTable();}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Initialize google DLP Clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Construct the tabular dataconsttablularData={headers:[{name:'AGE'},{name:'PATIENT'},{name:'HAPPINESS SCORE'},{name:'FACTOID'},],rows:[{values:[{integerValue:101},{stringValue:'Charles Dickens'},{integerValue:95},{stringValue:'Charles Dickens name was a curse invented by Shakespeare.',},],},{values:[{integerValue:22},{stringValue:'Jane Austen'},{integerValue:21},{stringValue:"There are 14 kisses in Jane Austen's novels."},],},{values:[{integerValue:55},{stringValue:'Mark Twain'},{integerValue:75},{stringValue:'Mark Twain loved cats.'},],},],};asyncfunctiondeIdentifyTableWithInfoType(){// Column that needs to be transformedconstfieldIds=[{name:'PATIENT'},{name:'FACTOID'}];// Construct InfoTypeTransformations configurationsconstinfoTypeTransformations={transformations:[{infoTypes:[{name:'PERSON_NAME'}],primitiveTransformation:{replaceWithInfoTypeConfig:{},},},],};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,item:{table:tablularData,},deidentifyConfig:{recordTransformations:{fieldTransformations:[{infoTypeTransformations,fields:fieldIds,},],},},};// Send the request and receive response from the serviceconst[response]=awaitdlp.deidentifyContent(request);// Print the resultsconsole.log(`Table after de-identification:${JSON.stringify(response.item.table)}`);}deIdentifyTableWithInfoType();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\InfoType;use Google\Cloud\Dlp\V2\InfoTypeTransformations;use Google\Cloud\Dlp\V2\InfoTypeTransformations\InfoTypeTransformation;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\ReplaceWithInfoTypeConfig;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data with infoTypes * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify * @param string $outputCsvFile The oupt file path to save deidentify content */function deidentify_table_infotypes( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table1.csv', string $outputCsvFile = './test/data/deidentify_table_infotypes_output.csv'): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify the content to be inspected. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify the type of info the inspection will look for. $personNameInfoType = (new InfoType()) ->setName('PERSON_NAME'); // Specify that findings should be replaced with corresponding info type name. $primitiveTransformation = (new PrimitiveTransformation()) ->setReplaceWithInfoTypeConfig(new ReplaceWithInfoTypeConfig()); // Associate info type with the replacement strategy $infoTypeTransformation = (new InfoTypeTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setInfoTypes([$personNameInfoType]); $infoTypeTransformations = (new InfoTypeTransformations()) ->setTransformations([$infoTypeTransformation]); // Specify fields to be de-identified. $fieldIds = [ (new FieldId())->setName('PATIENT'), (new FieldId())->setName('FACTOID'), ]; // Associate the de-identification and transformation with the specified fields. $fieldTransformation = (new FieldTransformation()) ->setInfoTypeTransformations($infoTypeTransformations) ->setFields($fieldIds); $recordtransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation]); $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Run request $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print the results $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf('After de-identify the table data (Output File Location): %s', $outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpdefdeidentify_table_replace_with_info_types(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],info_types:List[str],deid_content_list:List[str],)->None:""" Uses the Data Loss Prevention API to de-identify sensitive data in a table by replacing them with info type. Args: project: The Google Cloud project id to use as a parent resource. table_data: Json string representing table data. info_types: A list of strings representing info types to look for. A full list of info type categories can be fetched from the API. deid_content_list: A list of fields in table to de-identify Returns: None; the response from the API is printed to the terminal. Example: >> $ python deidentify_table_infotypes.py \ '{ "header": ["name", "email", "phone number"], "rows": [ ["Robert Frost", "robertfrost@example.com", "4232342345"], ["John Doe", "johndoe@example.com", "4253458383"] ] }' \ ["PERSON_NAME"] ["name"] >> '{ "header": ["name", "email", "phone number"], "rows": [ ["[PERSON_NAME]", "robertfrost@example.com", "4232342345"], ["[PERSON_NAME]", "johndoe@example.com", "4253458383"] ] }' """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct itemitem={"table":table}# Specify fields to be de-identifieddeid_content_list=[{"name":_i}for_iindeid_content_list]# Construct inspect configuration dictionaryinspect_config={"info_types":[{"name":info_type}forinfo_typeininfo_types]}# Construct deidentify configuration dictionarydeidentify_config={"record_transformations":{"field_transformations":[{"info_type_transformations":{"transformations":[{"primitive_transformation":{"replace_with_info_type_config":{}}}]},"fields":deid_content_list,}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"item":item,"inspect_config":inspect_config,})# Print the resultprint(f"Table after de-identification:{response.item.table}")"deidentifyConfig":{ "recordTransformations":{ "fieldTransformations":[ { "infoTypeTransformations":{ "transformations":[ { "infoTypes":[ { "name":"PERSON_NAME" } ], "primitiveTransformation":{ "replaceWithInfoTypeConfig":{ } } } ] }, "fields":[ { "name":"PATIENT" }, { "name":"FACTOID" } ] } ] }}Suppress a row based on the content of a column
You can remove a row entirely based on the content that appears in any column.This example suppresses the record for "Charles Dickens," as this patient isover 89 years old.
| Input | Transformed table | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, seeSensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyTableWithRowSuppress{publicstaticTableDeidentifyTable(stringprojectId,TabletableToInspect=null){// Instantiate a client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToInspect==null){varrow1=newValue[]{newValue{StringValue="101"},newValue{StringValue="Charles Dickens"},newValue{StringValue="95"}};varrow2=newValue[]{newValue{StringValue="22"},newValue{StringValue="Jane Austin"},newValue{StringValue="21"}};varrow3=newValue[]{newValue{StringValue="55"},newValue{StringValue="Mark Twain"},newValue{StringValue="75"}};tableToInspect=newTable{Headers={newFieldId{Name="AGE"},newFieldId{Name="PATIENT"},newFieldId{Name="HAPPINESS SCORE"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Construct the byte content item.varcontentItem=newContentItem{Table=tableToInspect};// Construct the conditions.varconditions=newRecordCondition.Types.Conditions{Conditions_={newRecordCondition.Types.Condition{Field=newFieldId{Name="AGE"},Operator=RelationalOperator.GreaterThan,Value=newValue{IntegerValue=89}}}};// Construct the deidentify config using the record suppression and conditions.vardeidentifyConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{RecordSuppressions={newRecordSuppression{Condition=newRecordCondition{Expressions=newRecordCondition.Types.Expressions{Conditions=conditions}}}}}};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifyConfig,Item=contentItem};// Call the API.varresponse=dlp.DeidentifyContent(request);// Inspect the response.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deidentifyTableRowSuppress de-identifies table data and// suppress a row based on the content of columnfuncdeidentifyTableRowSuppress(wio.Writer,projectIDstring)error{// projectId := "your-project-id"row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"22"}},{Type:&dlppb.Value_StringValue{StringValue:"Jane Austen"}},{Type:&dlppb.Value_StringValue{StringValue:"21"}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"55"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain"}},{Type:&dlppb.Value_StringValue{StringValue:"75"}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"101"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens"}},{Type:&dlppb.Value_StringValue{StringValue:"95"}},},}table:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"AGE"},{Name:"PATIENT"},{Name:"HAPPINESS SCORE"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:table,},}// Apply the condition to record suppression.condition:=&dlppb.RecordCondition{Expressions:&dlppb.RecordCondition_Expressions{Type:&dlppb.RecordCondition_Expressions_Conditions{Conditions:&dlppb.RecordCondition_Conditions{Conditions:[]*dlppb.RecordCondition_Condition{{Field:&dlppb.FieldId{Name:"AGE"},Operator:dlppb.RelationalOperator_GREATER_THAN,Value:&dlppb.Value{Type:&dlppb.Value_IntegerValue{IntegerValue:89},},},},},},},}recordSupression:=&dlppb.RecordSuppression{Condition:condition,}// Use record suppression as the only transformationrecordTransformations:=&dlppb.RecordTransformations{RecordSuppressions:[]*dlppb.RecordSuppression{recordSupression,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},},Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.RecordCondition;importcom.google.privacy.dlp.v2.RecordCondition.Condition;importcom.google.privacy.dlp.v2.RecordCondition.Conditions;importcom.google.privacy.dlp.v2.RecordCondition.Expressions;importcom.google.privacy.dlp.v2.RecordSuppression;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.RelationalOperator;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;publicclassDeIdentifyTableRowSuppress{publicstaticvoidmain(String[]args)throwsIOException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("AGE").build()).addHeaders(FieldId.newBuilder().setName("PATIENT").build()).addHeaders(FieldId.newBuilder().setName("HAPPINESS SCORE").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("101").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens").build()).addValues(Value.newBuilder().setStringValue("95").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("22").build()).addValues(Value.newBuilder().setStringValue("Jane Austen").build()).addValues(Value.newBuilder().setStringValue("21").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("55").build()).addValues(Value.newBuilder().setStringValue("Mark Twain").build()).addValues(Value.newBuilder().setStringValue("75").build()).build()).build();deIdentifyTableRowSuppress(projectId,tableToDeIdentify);}publicstaticTabledeIdentifyTableRowSuppress(StringprojectId,TabletableToDeIdentify)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to de-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify when the content should be de-identified.Conditioncondition=Condition.newBuilder().setField(FieldId.newBuilder().setName("AGE").build()).setOperator(RelationalOperator.GREATER_THAN).setValue(Value.newBuilder().setIntegerValue(89).build()).build();// Apply the condition to record suppression.RecordSuppressionrecordSuppressions=RecordSuppression.newBuilder().setCondition(RecordCondition.newBuilder().setExpressions(Expressions.newBuilder().setConditions(Conditions.newBuilder().addConditions(condition).build()).build()).build()).build();// Use record suppression as the only transformationRecordTransformationstransformations=RecordTransformations.newBuilder().addRecordSuppressions(recordSuppressions).build();DeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after de-identification: "+response.getItem().getTable());returnresponse.getItem().getTable();}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Initialize google DLP Clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Construct the tabular dataconsttablularData={headers:[{name:'AGE'},{name:'PATIENT'},{name:'HAPPINESS SCORE'}],rows:[{values:[{integerValue:101},{stringValue:'Charles Dickens'},{integerValue:95},],},{values:[{integerValue:22},{stringValue:'Jane Austen'},{integerValue:21},],},{values:[{integerValue:55},{stringValue:'Mark Twain'},{integerValue:75},],},],};asyncfunctiondeIdentifyTableRowSuppress(){// Specify when the content should be de-identified.constcondition={expressions:{conditions:{conditions:[{field:{name:'AGE'},operator:'GREATER_THAN',value:{integerValue:89},},],},},};// Apply the condition to record suppression.constrecordTransformations={recordSuppressions:[{condition,},],};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,item:{table:tablularData,},deidentifyConfig:{recordTransformations,},};// Send the request and receive response from the service.const[response]=awaitdlp.deidentifyContent(request);// Print the results.console.log(`Table after de-identification:${JSON.stringify(response.item.table)}`);}deIdentifyTableRowSuppress();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\RecordCondition;use Google\Cloud\Dlp\V2\RecordCondition\Condition;use Google\Cloud\Dlp\V2\RecordCondition\Conditions;use Google\Cloud\Dlp\V2\RecordCondition\Expressions;use Google\Cloud\Dlp\V2\RecordSuppression;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\RelationalOperator;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data: Suppress a row based on the content of a column * Suppress a row based on the content of a column. You can remove a row entirely based on the content that appears in any column. This example suppresses the record for "Charles Dickens," as this patient is over 89 years old. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify * @param string $outputCsvFile The oupt file path to save deidentify content */function deidentify_table_row_suppress( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table2.csv', string $outputCsvFile = './test/data/deidentify_table_row_suppress_output.csv'): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify what content you want the service to de-identify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify when the content should be de-identified. $condition = (new Condition()) ->setField((new FieldId()) ->setName('AGE')) ->setOperator(RelationalOperator::GREATER_THAN) ->setValue((new Value()) ->setIntegerValue(89)); // Apply the condition to record suppression. $recordSuppressions = (new RecordSuppression()) ->setCondition((new RecordCondition()) ->setExpressions((new Expressions()) ->setConditions((new Conditions()) ->setConditions([$condition]) ) ) ); // Use record suppression as the only transformation $recordtransformations = (new RecordTransformations()) ->setRecordSuppressions([$recordSuppressions]); // Create the deidentification configuration object $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Run request $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print the results $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf($outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpdefdeidentify_table_suppress_row(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],condition_field:str,condition_operator:str,condition_value:int,)->None:""" Uses the Data Loss Prevention API to de-identify sensitive data in a table by suppressing entire row/s based on a condition. Args: project: The Google Cloud project id to use as a parent resource. table_data: Dictionary representing table data. condition_field: A table field within the record this condition is evaluated against. condition_operator: Operator used to compare the field or infoType to the value. One of: RELATIONAL_OPERATOR_UNSPECIFIED, EQUAL_TO, NOT_EQUAL_TO, GREATER_THAN, LESS_THAN, GREATER_THAN_OR_EQUALS, LESS_THAN_OR_EQUALS, EXISTS. condition_value: Value to compare against. [Mandatory, except for ``EXISTS`` tests.]. Example: >> $ python deidentify_table_row_suppress.py \ '{"header": ["email", "phone number", "age"], "rows": [["robertfrost@example.com", "4232342345", "35"], ["johndoe@example.com", "4253458383", "64"]]}' \ "age" "GREATER_THAN" 50 >> '{"header": ["email", "phone number", "age"], "rows": [["robertfrost@example.com", "4232342345", "35", "21"]]}' """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the `item` containing the table data.item={"table":table}# Construct condition list.condition=[{"field":{"name":condition_field},"operator":condition_operator,"value":{"integer_value":condition_value},}]# Construct deidentify configuration dictionarydeidentify_config={"record_transformations":{"record_suppressions":[{"condition":{"expressions":{"conditions":{"conditions":condition}}}}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"item":item})# Print the result.print(f"Table after de-identification:{response.item.table}")"deidentifyConfig":{ "recordTransformations":{ "recordSuppressions":[ { "condition":{ "expressions":{ "conditions":{ "conditions":[ { "field":{ "name":"AGE" }, "operator":"GREATER_THAN", "value":{ "integerValue":"89" } } ] } } } } ] }}Transform findings only when specific conditions are met on another field
In this example, thePERSON_NAME findings are only redacted if the "AGE"column indicates the patient is over the age of 89.
| Input | Transformed table | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, seeSensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingSystem.Collections.Generic;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyTableUsingLogicInfoTypes{publicstaticTableDeidentify(stringprojectId,TabletableToInspect=null,IEnumerable<InfoType>infoTypes=null){// Instantiate the dlp client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToInspect==null){varrow1=newValue[]{newValue{StringValue="101"},newValue{StringValue="Charles Dickens"},newValue{StringValue="95"},newValue{StringValue="Charles Dickens name was a curse invented by Shakespeare."}};varrow2=newValue[]{newValue{StringValue="22"},newValue{StringValue="Jane Austin"},newValue{StringValue="21"},newValue{StringValue="There are 14 kisses in Jane Austen's novels."}};varrow3=newValue[]{newValue{StringValue="55"},newValue{StringValue="Mark Twain"},newValue{StringValue="75"},newValue{StringValue="Mark Twain loved cats."}};tableToInspect=newTable{Headers={newFieldId{Name="AGE"},newFieldId{Name="PATIENT"},newFieldId{Name="HAPPINESS SCORE"},newFieldId{Name="FACTOID"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Construct the table content item.varcontentItem=newContentItem{Table=tableToInspect};// Specify that findings should be replaced with corresponding info type name.varinfoTypeTransformation=newInfoTypeTransformations{Transformations={newInfoTypeTransformations.Types.InfoTypeTransformation{InfoTypes={infoTypes??newInfoType[]{newInfoType{Name="PERSON_NAME"}}},PrimitiveTransformation=newPrimitiveTransformation{ReplaceWithInfoTypeConfig=newReplaceWithInfoTypeConfig()}}}};// Specify the fields to be de-identified.varfields=newFieldId[]{newFieldId{Name="PATIENT"},newFieldId{Name="FACTOID"}};// Specify when the above fields should be de-identified using condition.varcondition=newRecordCondition.Types.Conditions{Conditions_={newRecordCondition.Types.Condition{Field=newFieldId{Name="AGE"},Operator=RelationalOperator.GreaterThan,Value=newValue{IntegerValue=89}}}};// Apply the condition to records.varrecordCondition=newRecordCondition{Expressions=newRecordCondition.Types.Expressions{Conditions=condition}};// Associate the de-identification and conditions with the specified fields.vardeidentifiedConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={newFieldTransformation{InfoTypeTransformations=infoTypeTransformation,Fields={fields},Condition=recordCondition}}}};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifiedConfig,Item=contentItem};// Call the API.varresponse=dlp.DeidentifyContent(request);// Inspect the response.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deidentifyTableConditionInfoTypes de-identifies table data// using conditional logic and replace with infoTypesfuncdeidentifyTableConditionInfoTypes(wio.Writer,projectIDstring,columnNames[]string)error{// projectId := "your-project-id"// columnNames := []string{"PATIENT", "FACTOID"}row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"22"}},{Type:&dlppb.Value_StringValue{StringValue:"Jane Austen"}},{Type:&dlppb.Value_StringValue{StringValue:"21"}},{Type:&dlppb.Value_StringValue{StringValue:"There are 14 kisses in Jane Austen's novels."}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"55"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain"}},{Type:&dlppb.Value_StringValue{StringValue:"75"}},{Type:&dlppb.Value_StringValue{StringValue:"Mark Twain loved cats."}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"101"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens"}},{Type:&dlppb.Value_StringValue{StringValue:"95"}},{Type:&dlppb.Value_StringValue{StringValue:"Charles Dickens name was a curse invented by Shakespeare."}},},}table:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"AGE"},{Name:"PATIENT"},{Name:"HAPPINESS SCORE"},{Name:"FACTOID"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:table,},}// Specify how the content should be de-identified.// Select type of info to be replaced.infoTypes:=[]*dlppb.InfoType{{Name:"PERSON_NAME"},}// Specify that findings should be replaced with corresponding info type name.primitiveTransformation:=&dlppb.PrimitiveTransformation{Transformation:&dlppb.PrimitiveTransformation_ReplaceWithInfoTypeConfig{},}// Associate info type with the replacement strategyinfoTypeTransformation:=&dlppb.InfoTypeTransformations_InfoTypeTransformation{InfoTypes:infoTypes,PrimitiveTransformation:primitiveTransformation,}infoTypeTransformations:=&dlppb.InfoTypeTransformations{Transformations:[]*dlppb.InfoTypeTransformations_InfoTypeTransformation{infoTypeTransformation,},}// Specify fields to be de-identified.varf[]*dlppb.FieldIdfor_,c:=rangecolumnNames{f=append(f,&dlppb.FieldId{Name:c})}// Specify when the above fields should be de-identified.condition:=&dlppb.RecordCondition{Expressions:&dlppb.RecordCondition_Expressions{Type:&dlppb.RecordCondition_Expressions_Conditions{Conditions:&dlppb.RecordCondition_Conditions{Conditions:[]*dlppb.RecordCondition_Condition{{Field:&dlppb.FieldId{Name:"AGE",},Operator:dlppb.RelationalOperator_GREATER_THAN,Value:&dlppb.Value{Type:&dlppb.Value_IntegerValue{IntegerValue:89,},},},},},},},}// Associate the de-identification and conditions with the specified fields.fieldTransformation:=&dlppb.FieldTransformation{Fields:f,Condition:condition,Transformation:&dlppb.FieldTransformation_InfoTypeTransformations{InfoTypeTransformations:infoTypeTransformations,},}recordTransformations:=&dlppb.RecordTransformations{FieldTransformations:[]*dlppb.FieldTransformation{fieldTransformation,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},},Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.InfoType;importcom.google.privacy.dlp.v2.InfoTypeTransformations;importcom.google.privacy.dlp.v2.InfoTypeTransformations.InfoTypeTransformation;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordCondition;importcom.google.privacy.dlp.v2.RecordCondition.Condition;importcom.google.privacy.dlp.v2.RecordCondition.Conditions;importcom.google.privacy.dlp.v2.RecordCondition.Expressions;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.RelationalOperator;importcom.google.privacy.dlp.v2.ReplaceWithInfoTypeConfig;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;importjava.util.List;importjava.util.stream.Collectors;importjava.util.stream.Stream;publicclassDeIdentifyTableConditionInfoTypes{publicstaticvoidmain(String[]args)throwsIOException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("AGE").build()).addHeaders(FieldId.newBuilder().setName("PATIENT").build()).addHeaders(FieldId.newBuilder().setName("HAPPINESS SCORE").build()).addHeaders(FieldId.newBuilder().setName("FACTOID").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("101").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens").build()).addValues(Value.newBuilder().setStringValue("95").build()).addValues(Value.newBuilder().setStringValue("Charles Dickens name was a curse invented by Shakespeare.").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("22").build()).addValues(Value.newBuilder().setStringValue("Jane Austen").build()).addValues(Value.newBuilder().setStringValue("21").build()).addValues(Value.newBuilder().setStringValue("There are 14 kisses in Jane Austen's novels.").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("55").build()).addValues(Value.newBuilder().setStringValue("Mark Twain").build()).addValues(Value.newBuilder().setStringValue("75").build()).addValues(Value.newBuilder().setStringValue("Mark Twain loved cats.").build()).build()).build();deIdentifyTableConditionInfoTypes(projectId,tableToDeIdentify);}publicstaticTabledeIdentifyTableConditionInfoTypes(StringprojectId,TabletableToDeIdentify)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to de-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify how the content should be de-identified.// Select type of info to be replaced.InfoTypeinfoType=InfoType.newBuilder().setName("PERSON_NAME").build();// Specify that findings should be replaced with corresponding info type name.ReplaceWithInfoTypeConfigreplaceWithInfoTypeConfig=ReplaceWithInfoTypeConfig.getDefaultInstance();PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setReplaceWithInfoTypeConfig(replaceWithInfoTypeConfig).build();// Associate info type with the replacement strategyInfoTypeTransformationinfoTypeTransformation=InfoTypeTransformation.newBuilder().addInfoTypes(infoType).setPrimitiveTransformation(primitiveTransformation).build();InfoTypeTransformationsinfoTypeTransformations=InfoTypeTransformations.newBuilder().addTransformations(infoTypeTransformation).build();// Specify fields to be de-identified.List<FieldId>fieldIds=Stream.of("PATIENT","FACTOID").map(id->FieldId.newBuilder().setName(id).build()).collect(Collectors.toList());// Specify when the above fields should be de-identified.Conditioncondition=Condition.newBuilder().setField(FieldId.newBuilder().setName("AGE").build()).setOperator(RelationalOperator.GREATER_THAN).setValue(Value.newBuilder().setIntegerValue(89).build()).build();// Apply the condition to recordsRecordConditionrecordCondition=RecordCondition.newBuilder().setExpressions(Expressions.newBuilder().setConditions(Conditions.newBuilder().addConditions(condition).build()).build()).build();// Associate the de-identification and conditions with the specified fields.FieldTransformationfieldTransformation=FieldTransformation.newBuilder().setInfoTypeTransformations(infoTypeTransformations).addAllFields(fieldIds).setCondition(recordCondition).build();RecordTransformationstransformations=RecordTransformations.newBuilder().addFieldTransformations(fieldTransformation).build();DeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after de-identification: "+response.getItem().getTable());returnresponse.getItem().getTable();}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Initialize google DLP Clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Construct the tabular dataconsttablularData={headers:[{name:'AGE'},{name:'PATIENT'},{name:'HAPPINESS SCORE'},{name:'FACTOID'},],rows:[{values:[{integerValue:101},{stringValue:'Charles Dickens'},{integerValue:95},{stringValue:'Charles Dickens name was a curse invented by Shakespeare.',},],},{values:[{integerValue:22},{stringValue:'Jane Austen'},{integerValue:21},{stringValue:"There are 14 kisses in Jane Austen's novels."},],},{values:[{integerValue:55},{stringValue:'Mark Twain'},{integerValue:75},{stringValue:'Mark Twain loved cats.'},],},],};asyncfunctiondeIdentifyTableConditionalInfoType(){// Specify fields to be de-identified.constfieldIds=[{name:'PATIENT'},{name:'FACTOID'}];// Associate info type with the replacement strategyconstinfoTypeTransformations={transformations:[{infoTypes:[{name:'PERSON_NAME'}],primitiveTransformation:{replaceWithInfoTypeConfig:{},},},],};// Specify when the above fields should be de-identified.constcondition={expressions:{conditions:{conditions:[{field:{name:'AGE'},operator:'GREATER_THAN',value:{integerValue:89},},],},},};// Apply the condition to records.constrecordTransformations={fieldTransformations:[{infoTypeTransformations,fields:fieldIds,condition:condition,},],};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,item:{table:tablularData,},deidentifyConfig:{recordTransformations,},};// Send the request and receive response from the service.const[response]=awaitdlp.deidentifyContent(request);// Print the results.console.log(`Table after de-identification:${JSON.stringify(response.item.table)}`);}deIdentifyTableConditionalInfoType();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\InfoType;use Google\Cloud\Dlp\V2\InfoTypeTransformations;use Google\Cloud\Dlp\V2\InfoTypeTransformations\InfoTypeTransformation;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordCondition;use Google\Cloud\Dlp\V2\RecordCondition\Condition;use Google\Cloud\Dlp\V2\RecordCondition\Conditions;use Google\Cloud\Dlp\V2\RecordCondition\Expressions;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\RelationalOperator;use Google\Cloud\Dlp\V2\ReplaceWithInfoTypeConfig;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data using conditional logic and replace with infoTypes. * Transform findings only when specific conditions are met on another field. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify * @param string $outputCsvFile The oupt file path to save deidentify content */function deidentify_table_condition_infotypes( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table1.csv', string $outputCsvFile = './test/data/deidentify_table_condition_infotypes_output.csv'): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify what content you want the service to de-identify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify the type of info the inspection will look for. $personNameInfoType = (new InfoType()) ->setName('PERSON_NAME'); // Specify that findings should be replaced with corresponding info type name. $primitiveTransformation = (new PrimitiveTransformation()) ->setReplaceWithInfoTypeConfig(new ReplaceWithInfoTypeConfig()); // Associate info type with the replacement strategy $infoTypeTransformation = (new InfoTypeTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setInfoTypes([$personNameInfoType]); $infoTypeTransformations = (new InfoTypeTransformations()) ->setTransformations([$infoTypeTransformation]); // Specify fields to be de-identified. $fieldIds = [ (new FieldId())->setName('PATIENT'), (new FieldId())->setName('FACTOID'), ]; // Specify when the above fields should be de-identified. $condition = (new Condition()) ->setField((new FieldId()) ->setName('AGE')) ->setOperator(RelationalOperator::GREATER_THAN) ->setValue((new Value()) ->setIntegerValue(89)); // Apply the condition to records $recordCondition = (new RecordCondition()) ->setExpressions((new Expressions()) ->setConditions((new Conditions()) ->setConditions([$condition]) ) ); // Associate the de-identification and conditions with the specified fields. $fieldTransformation = (new FieldTransformation()) ->setInfoTypeTransformations($infoTypeTransformations) ->setFields($fieldIds) ->setCondition($recordCondition); $recordtransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation]); $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Run request $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print results $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf($outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpfromgoogle.cloud.dlp_v2importtypesdefdeidentify_table_condition_replace_with_info_types(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],deid_content_list:List[str],info_types:List[str],condition_field:str=None,condition_operator:str=None,condition_value:int=None,)->types.dlp.Table:"""Uses the Data Loss Prevention API to de-identify sensitive data in a table by replacing them with info-types based on a condition. Args: project: The Google Cloud project id to use as a parent resource. table_data: Json string representing table data. deid_content_list: A list of fields in table to de-identify. info_types: A list of strings representing info types to look for. A full list of info categories and types is available from the API. Examples include "FIRST_NAME", "LAST_NAME", "EMAIL_ADDRESS". ' condition_field: A table field within the record this condition is evaluated against. condition_operator: Operator used to compare the field or infoType to the value. One of: RELATIONAL_OPERATOR_UNSPECIFIED, EQUAL_TO, NOT_EQUAL_TO, GREATER_THAN, LESS_THAN, GREATER_THAN_OR_EQUALS, LESS_THAN_OR_EQUALS, EXISTS. condition_value: Value to compare against. [Mandatory, except for ``EXISTS`` tests.]. Returns: De-identified table is returned; the response from the API is also printed to the terminal. Example: >> $ python deidentify_table_condition_infotypes.py \ '{"header": ["email", "phone number", "age"], "rows": [["robertfrost@example.com", "4232342345", "45"], ["johndoe@example.com", "4253458383", "63"]]}' ["email"] \ ["EMAIL_ADDRESS"] "age" "GREATER_THAN" 50 >> '{"header": ["email", "phone number", "age"], "rows": [["robertfrost@example.com", "4232342345", "45"], ["[EMAIL_ADDRESS]", "4253458383", "63"]]}' """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the itemitem={"table":table}# Specify fields to be de-identifieddeid_field_list=[{"name":_i}for_iindeid_content_list]# Construct inspect configuration dictionaryinspect_config={"info_types":[{"name":info_type}forinfo_typeininfo_types]}# Construct condition listcondition=[{"field":{"name":condition_field},"operator":condition_operator,"value":{"integer_value":condition_value},}]# Construct deidentify configuration dictionarydeidentify_config={"record_transformations":{"field_transformations":[{"info_type_transformations":{"transformations":[{"primitive_transformation":{"replace_with_info_type_config":{}}}]},"fields":deid_field_list,"condition":{"expressions":{"conditions":{"conditions":condition}}},}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"item":item,"inspect_config":inspect_config,})print(f"Table after de-identification:{response.item.table}")returnresponse.item.table"deidentifyConfig":{ "recordTransformations":{ "fieldTransformations":[ { "infoTypeTransformations":{ "transformations":[ { "infoTypes":[ { "name":"PERSON_NAME" } ], "primitiveTransformation":{ "replaceWithInfoTypeConfig":{ } } } ] }, "fields":[ { "name":"PATIENT" }, { "name":"FACTOID" } ], "condition":{ "expressions":{ "conditions":{ "conditions":[ { "field":{ "name":"AGE" }, "operator":"GREATER_THAN", "value":{ "integerValue":"89" } } ] } } } } ] }}Transform findings using a cryptographic hash transformation
The following JSON examples useinfoTypetransformationsto instruct the DLP API to inspect the entire table structurefor specificinfoTypes, and then to encrypt the matchingvalues using atransientCryptoKey.
CryptoKeys,though using a rawUnwrappedCryptoKeyis less secure and not recommended.The following example demonstrates de-identifying two infoTypes using acryptographic hash transformation.
Input:
| userid | comments |
|---|---|
| user1@example.org | my email is user1@example.org and phone is 858-555-0222 |
| user2@example.org | my email is user2@example.org and phone is 858-555-0223 |
| user3@example.org | my email is user3@example.org and phone is 858-555-0224 |
Transformed table:
| userid | comments |
|---|---|
| 1kSfj3Op64MH1BiznupEpX0BdQrHMm62X6abgsPH5zM= | my email is 1kSfj3Op64MH1BiznupEpX0BdQrHMm62X6abgsPH5zM= and phone is hYXPcsJNBCe1rr51sHiVw2KhtoyMe4HEFKNHWFcDVm0= |
| 4ESy7+rEN8NVaUJ6J7kwvcgW8wcm0cm5gbBAcu6SfdM= | my email is 4ESy7+rEN8NVaUJ6J7kwvcgW8wcm0cm5gbBAcu6SfdM= and phone is KKqW1tQwgvGiC6iWJHhLiz2enNSEFRzhmLOf9fSTxRw= |
| bu1blyd/mbjLmpF2Rdi6zpgsLatSwpJLVki2fMeudM0= | my email is bu1blyd/mbjLmpF2Rdi6zpgsLatSwpJLVki2fMeudM0= and phone is eNt7qtZVLmxRb8z8NBR/+z00In07CI3hEMStbwofWoc= |
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingSystem.Collections.Generic;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyTableWithCryptoHash{publicstaticTableDeidentify(stringprojectId,TabletableToDeidentify=null,IEnumerable<InfoType>infoTypes=null,stringtransientKeyName=null){// Instantiate the client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToDeidentify==null){varrow1=newValue[]{newValue{StringValue="user1@example.org"},newValue{StringValue="my email is user1@example.org and phone is 858-555-0222"}};varrow2=newValue[]{newValue{StringValue="user2@example.org"},newValue{StringValue="my email is user2@example.org and phone is 858-555-0223"}};varrow3=newValue[]{newValue{StringValue="user3@example.org"},newValue{StringValue="my email is user3@example.org and phone is 858-555-0224"}};tableToDeidentify=newTable{Headers={newFieldId{Name="User ID"},newFieldId{Name="comments"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Specify the table and construct the content item.varcontentItem=newContentItem{Table=tableToDeidentify};// Construct the infoTypes by specifying the type of info to be inspected if null.varinfotypes=infoTypes??newInfoType[]{newInfoType{Name="EMAIL_ADDRESS"},newInfoType{Name="PHONE_NUMBER"}};// Construct the crypto hash config using transient crypto key name.varcryptoHashConfig=newCryptoHashConfig{CryptoKey=newCryptoKey{Transient=newTransientCryptoKey{Name=transientKeyName??"[TRANSIENT-CRYPTO-KEY]"}}};// Construct the de-identify config using crypto hash config.vardeidentifyConfig=newDeidentifyConfig{InfoTypeTransformations=newInfoTypeTransformations{Transformations={newInfoTypeTransformations.Types.InfoTypeTransformation{PrimitiveTransformation=newPrimitiveTransformation{CryptoHashConfig=cryptoHashConfig},InfoTypes={infotypes}}}}};// Construct the inspect config.varinspectConfig=newInspectConfig{InfoTypes={infotypes},IncludeQuote=true};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifyConfig,Item=contentItem,InspectConfig=inspectConfig};// Call the API.DeidentifyContentResponseresponse=dlp.DeidentifyContent(request);// Print the table.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deIdentifyTableWithCryptoHash transforms findings using a cryptographic hash transformation.funcdeIdentifyTableWithCryptoHash(wio.Writer,projectID,transientKeyNamestring)error{// projectId := "your-project-id"// transientKeyName := "YOUR_TRANSIENT_CRYPTO_KEY_NAME"row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"user1@example.org"}},{Type:&dlppb.Value_StringValue{StringValue:"my email is user1@example.org and phone is 858-555-0222"}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"user2@example.org"}},{Type:&dlppb.Value_StringValue{StringValue:"my email is user2@example.org and phone is 858-555-0232"}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"user3@example.org"}},{Type:&dlppb.Value_StringValue{StringValue:"my email is user3@example.org and phone is 858-555-0224"}},},}tableToDeidentify:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"userid"},{Name:"comments"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:tableToDeidentify,},}// Specify the type of info the inspection will look for.// See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info typesinfoTypes:=[]*dlppb.InfoType{{Name:"PHONE_NUMBER"},{Name:"EMAIL_ADDRESS"},}inspectConfig:=&dlppb.InspectConfig{InfoTypes:infoTypes,}// Specify the transient key which will encrypt the data.iftransientKeyName==""{transientKeyName="YOUR_TRANSIENT_CRYPTO_KEY_NAME"}// Specify the transient key which will encrypt the data.cryptoKey:=&dlppb.CryptoKey{Source:&dlppb.CryptoKey_Transient{Transient:&dlppb.TransientCryptoKey{Name:transientKeyName,},},}// Specify how the info from the inspection should be encrypted.cryptoHashConfig:=&dlppb.CryptoHashConfig{CryptoKey:cryptoKey,}// Define type of de-identification as cryptographic hash transformation.primitiveTransformation:=&dlppb.PrimitiveTransformation_CryptoHashConfig{CryptoHashConfig:cryptoHashConfig,}infoTypeTransformation:=&dlppb.InfoTypeTransformations_InfoTypeTransformation{InfoTypes:infoTypes,PrimitiveTransformation:&dlppb.PrimitiveTransformation{Transformation:primitiveTransformation,},}transformations:=&dlppb.InfoTypeTransformations{Transformations:[]*dlppb.InfoTypeTransformations_InfoTypeTransformation{infoTypeTransformation,},}// Specify the config for the de-identify request.deidentifyConfig:=&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_InfoTypeTransformations{InfoTypeTransformations:transformations,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:deidentifyConfig,InspectConfig:inspectConfig,Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.CryptoHashConfig;importcom.google.privacy.dlp.v2.CryptoKey;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.InfoType;importcom.google.privacy.dlp.v2.InfoTypeTransformations;importcom.google.privacy.dlp.v2.InspectConfig;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.TransientCryptoKey;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;importjava.util.List;importjava.util.stream.Collectors;importjava.util.stream.Stream;publicclassDeIdentifyTableWithCryptoHash{publicstaticvoidmain(String[]args)throwsException{// TODO(developer): Replace these variables before running the sample.// The Google Cloud project id to use as a parent resource.StringprojectId="your-project-id";// The table to de-identify.TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("userid").build()).addHeaders(FieldId.newBuilder().setName("comments").build()).addRows(Table.Row.newBuilder().addValues(Value.newBuilder().setStringValue("user1@example.org").build()).addValues(Value.newBuilder().setStringValue("my email is user1@example.org and phone is 858-555-0222").build()).build()).addRows(Table.Row.newBuilder().addValues(Value.newBuilder().setStringValue("user2@example.org").build()).addValues(Value.newBuilder().setStringValue("my email is user2@example.org and phone is 858-555-0223").build()).build()).addRows(Table.Row.newBuilder().addValues(Value.newBuilder().setStringValue("user3@example.org").build()).addValues(Value.newBuilder().setStringValue("my email is user3@example.org and phone is 858-555-0224").build()).build()).build();// The randomly generated crypto key to encrypt the data.StringtransientKeyName="YOUR_TRANSIENT_CRYPTO_KEY";deIdentifyWithCryptHashTransformation(projectId,tableToDeIdentify,transientKeyName);}// Transforms findings using a cryptographic hash transformation.publicstaticvoiddeIdentifyWithCryptHashTransformation(StringprojectId,TabletableToDeIdentify,StringtransientKeyName)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to DeIdentifyContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify the type of info the inspection will look for.// See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info typesList<InfoType>infoTypes=Stream.of("PHONE_NUMBER","EMAIL_ADDRESS").map(it->InfoType.newBuilder().setName(it).build()).collect(Collectors.toList());InspectConfiginspectConfig=InspectConfig.newBuilder().addAllInfoTypes(infoTypes).build();// Specify the transient key which will encrypt the data.TransientCryptoKeytransientCryptoKey=TransientCryptoKey.newBuilder().setName(transientKeyName).build();CryptoKeycryptoKey=CryptoKey.newBuilder().setTransient(transientCryptoKey).build();// Specify how the info from the inspection should be encrypted.CryptoHashConfigcryptoHashConfig=CryptoHashConfig.newBuilder().setCryptoKey(cryptoKey).build();// Define type of de-identification as cryptographic hash transformation.PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setCryptoHashConfig(cryptoHashConfig).build();InfoTypeTransformations.InfoTypeTransformationinfoTypeTransformation=InfoTypeTransformations.InfoTypeTransformation.newBuilder().setPrimitiveTransformation(primitiveTransformation).addAllInfoTypes(infoTypes).build();InfoTypeTransformationstransformations=InfoTypeTransformations.newBuilder().addTransformations(infoTypeTransformation).build();// Specify the config for the de-identify requestDeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setInfoTypeTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setInspectConfig(inspectConfig).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the serviceDeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the resultsSystem.out.println("Table after de-identification: "+response.getItem().getTable());}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Instantiates a clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Crypto key// const transientKeyName = 'YOUR_TRANSIENT_CRYPTO_KEY';// The table to de-identify.consttableToDeIdentify={headers:[{name:'userid'},{name:'comments'}],rows:[{values:[{stringValue:'user1@example.org'},{stringValue:'my email is user1@example.org and phone is 858-555-0222',},],},{values:[{stringValue:'user2@example.org'},{stringValue:'my email is user2@example.org and phone is 858-555-0223',},],},{values:[{stringValue:'user3@example.org'},{stringValue:'my email is user3@example.org and phone is 858-555-0224',},],},],};asyncfunctiondeIdentifyTableWithCryptoHash(){// Specify crypto hash configuration that uses transient key.constcryptoHashConfig={cryptoKey:{transient:{name:transientKeyName,},},};// Construct de-identify request that uses crypto hash configuration.constrequest={parent:`projects/${projectId}/locations/global`,deidentifyConfig:{infoTypeTransformations:{transformations:[{primitiveTransformation:{cryptoHashConfig:cryptoHashConfig,},infoTypes:[{name:'PHONE_NUMBER'},{name:'EMAIL_ADDRESS'}],},],},},item:{table:tableToDeIdentify},};// Send the request and receive response from the service.const[response]=awaitdlp.deidentifyContent(request);constdeidentifiedTable=response.item.table;// Print the results.console.log(`Table after de-identification:\n${JSON.stringify(deidentifiedTable,null,2)}`);}deIdentifyTableWithCryptoHash();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\CryptoHashConfig;use Google\Cloud\Dlp\V2\CryptoKey;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\InfoType;use Google\Cloud\Dlp\V2\InfoTypeTransformations;use Google\Cloud\Dlp\V2\InfoTypeTransformations\InfoTypeTransformation;use Google\Cloud\Dlp\V2\InspectConfig;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\TransientCryptoKey;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data with crypto hash. * Transform findings using a cryptographic hash transformation. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify. * @param string $outputCsvFile The oupt file path to save deidentify content. * @param string $transientCryptoKeyName Specify the random string. */function deidentify_table_with_crypto_hash( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table5.csv', string $outputCsvFile = './test/data/deidentify_table_with_crypto_hash_output.csv', string $transientCryptoKeyName = 'YOUR-TRANSIENT-CRYPTO-KEY'): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file. $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects. $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object. $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify what content you want the service to de-identify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify the type of info the inspection will look for. // See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info types $infoTypes = [ (new InfoType())->setName('EMAIL_ADDRESS'), (new InfoType())->setName('PHONE_NUMBER') ]; $inspectConfig = (new InspectConfig()) ->setInfoTypes($infoTypes); // Specify the transient key which will encrypt the data. $cryptoKey = (new CryptoKey()) ->setTransient((new TransientCryptoKey()) ->setName($transientCryptoKeyName)); // Specify how the info from the inspection should be encrypted. $cryptoHashConfig = (new CryptoHashConfig()) ->setCryptoKey($cryptoKey); // Define type of de-identification as cryptographic hash transformation. $primitiveTransformation = (new PrimitiveTransformation()) ->setCryptoHashConfig($cryptoHashConfig); $infoTypeTransformation = (new InfoTypeTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setInfoTypes($infoTypes); $infoTypeTransformations = (new InfoTypeTransformations()) ->setTransformations([$infoTypeTransformation]); // Specify the config for the de-identify request $deidentifyConfig = (new DeidentifyConfig()) ->setInfoTypeTransformations($infoTypeTransformations); // Send the request and receive response from the service. $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setInspectConfig($inspectConfig) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print the results. $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf('Table after deidentify (File Location): %s', $outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpdefdeidentify_table_with_crypto_hash(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],info_types:List[str],transient_key_name:str,)->None:"""Uses the Data Loss Prevention API to de-identify sensitive data in a table using a cryptographic hash transformation. Args: project: The Google Cloud project id to use as a parent resource. table_data: Dictionary representing table data. info_types: A list of strings representing info types to look for. A full list of info type categories can be fetched from the API. transient_key_name: Name of the transient crypto key used for encryption. The scope of this key is a single API call. It is generated for the transformation and then discarded. """# Instantiate a clientdlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the `item` that service will de-identify.item={"table":table}# Prepare info_types by converting the list of strings into a list of# dictionaries.info_types=[{"name":info_type}forinfo_typeininfo_types]# Construct cryptographic hash configuration using the transient key# which will encrypt the data.crypto_hash_config={"crypto_key":{"transient":{"name":transient_key_name}}}# Specify the type of info the inspection will look for.inspect_config={"info_types":info_types,}# Construct deidentify configuration dictionary.deidentify_config={"info_type_transformations":{"transformations":[{"info_types":info_types,"primitive_transformation":{"crypto_hash_config":crypto_hash_config},}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"inspect_config":inspect_config,"item":item,})# Print the result.print(f"Table after de-identification:{response.item.table}")REST
{ "inspectConfig":{ "infoTypes":[ { "name":"EMAIL_ADDRESS" }, { "name":"PHONE_NUMBER" } ] }, "deidentifyConfig":{ "infoTypeTransformations":{ "transformations":[ { "infoTypes":[ { "name":"EMAIL_ADDRESS" }, { "name":"PHONE_NUMBER" } ], "primitiveTransformation":{ "cryptoHashConfig":{ "cryptoKey":{ "transient":{ "name":"[TRANSIENT-CRYPTO-KEY]" } } } } } ] } }, "item":{ "table":{ "headers":[ { "name":"userid" }, { "name":"comments" } ], "rows":[ { "values":[ { "stringValue":"abby_abernathy@example.org" }, { "stringValue":"my email is abby_abernathy@example.org and phone is 858-555-0222" } ] }, { "values":[ { "stringValue":"bert_beauregard@example.org" }, { "stringValue":"my email is bert_beauregard@example.org and phone is 858-555-0223" } ] }, { "values":[ { "stringValue":"cathy_crenshaw@example.org" }, { "stringValue":"my email is cathy_crenshaw@example.org and phone is 858-555-0224" } ] } ] } } }Transform findings using two separate cryptographic hash transformations
This example demonstrates how you can use different cryptographic keys indifferent transformations within a single de-identification configuration.First, a field transformation on the "userid" field is declared. Thattransformation does not include any infoType transformations, so the "userid"field in each row is transformed, regardless of its data type. Then, anotherfield transformation is declared, this one on the "comments" field.
Input:
| userid | comments |
|---|---|
| user1@example.org | my email is user1@example.org and phone is 858-555-0222 |
| abbyabernathy1 | my userid is abbyabernathy1 and my email is aabernathy@example.com |
Transformed table:
| userid | comments |
|---|---|
| 5WvS4+aJtCCwWWG79cmRNamDgyvJ+CkuwNpA2gaR1VQ= | my email is vjqGLaA6+NUUnZAWXpI72lU1GfwQdOKu7XqWaJPcvQQ= and phone is BY+mSXXTu6mOoX5pr0Xbse60uelsSHmwRCq6HcscKtk= |
| t0dOmHvkT0VsM++SVmESVKHenLkmhBmFezH3hSDldDg= | my userid is abbyabernathy1 and my email is TQ3ancdUn9zgwO5qe6ahkmVrBuNhvlMknxjPjIt0N2w= |
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingSystem.Collections.Generic;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;publicclassDeidentifyTableWithMultipleCryptoHash{publicstaticTableDeidentify(stringprojectId,TabletableToDeidentify=null,IEnumerable<InfoType>infoTypes=null,stringtransientKeyName1=null,stringtransientKeyName2=null){// Instantiate the client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToDeidentify==null){varrow1=newValue[]{newValue{StringValue="user1@example.org"},newValue{StringValue="my email is user1@example.org and phone is 858-555-0222"}};varrow2=newValue[]{newValue{StringValue="user2@example.org"},newValue{StringValue="my email is user2@example.org and phone is 858-555-0223"}};varrow3=newValue[]{newValue{StringValue="user3@example.org"},newValue{StringValue="my email is user3@example.org and phone is 858-555-0224"}};tableToDeidentify=newTable{Headers={newFieldId{Name="User ID"},newFieldId{Name="comments"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Specify the table and construct the content item.varcontentItem=newContentItem{Table=tableToDeidentify};// Construct the crypto hash config for primitive transformation using// transient crypto key name.varcryptoHashConfig1=newCryptoHashConfig{CryptoKey=newCryptoKey{Transient=newTransientCryptoKey{Name=transientKeyName1??"[TRANSIENT-CRYPTO-KEY-1]"}}};// Construct the crypto hash config for infoType transformation using// transient crypto key name.varcryptoHashConfig2=newCryptoHashConfig{CryptoKey=newCryptoKey{Transient=newTransientCryptoKey{Name=transientKeyName2??"[TRANSIENT-CRYPTO-KEY-2]"}}};// Construct the infoTypes by specifying the type of info to be inspected if null.varinfotypes=infoTypes??newInfoType[]{newInfoType{Name="EMAIL_ADDRESS"},newInfoType{Name="PERSON_NAME"}};// Construct the deidentify config using crypto hash configs.vardeidentifyConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={newFieldTransformation{Fields={newFieldId[]{newFieldId{Name="User ID"}}},PrimitiveTransformation=newPrimitiveTransformation{CryptoHashConfig=cryptoHashConfig1}},newFieldTransformation{Fields={newFieldId[]{newFieldId{Name="comments"}}},InfoTypeTransformations=newInfoTypeTransformations{Transformations={newInfoTypeTransformations.Types.InfoTypeTransformation{PrimitiveTransformation=newPrimitiveTransformation{CryptoHashConfig=cryptoHashConfig2},InfoTypes={infotypes}}}}}}}};// Construct the inspect config.varinspectConfig=newInspectConfig{InfoTypes={infotypes},IncludeQuote=true};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifyConfig,Item=contentItem,InspectConfig=inspectConfig};// Call the API.varresponse=dlp.DeidentifyContent(request);// Print the table.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deIdentifyTableWithMultipleCryptoHash Transforms findings using two separate cryptographic hash transformations.funcdeIdentifyTableWithMultipleCryptoHash(wio.Writer,projectID,transientKeyName1,transientKeyName2string)error{// projectId := "your-project-id"// transientKeyName1 := "YOUR_TRANSIENT_CRYPTO_KEY_1"// transientKeyName2 := "YOUR_TRANSIENT_CRYPTO_KEY_2"// The table to de-identify.row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"user1@example.org"}},{Type:&dlppb.Value_StringValue{StringValue:"my email is user1@example.org and phone is 858-555-0222"}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"abbyabernathy1"}},{Type:&dlppb.Value_StringValue{StringValue:"my userid is abbyabernathy1 and my email is aabernathy@example.com"}},},}tableToDeidentify:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"userid"},{Name:"comments"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:tableToDeidentify,},}// Specify the type of info the inspection will look for.// See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info typesinfoTypes:=[]*dlppb.InfoType{{Name:"PHONE_NUMBER"},{Name:"EMAIL_ADDRESS"},}inspectConfig:=&dlppb.InspectConfig{InfoTypes:infoTypes,}// Specify the transient key names.transientCryptoKey1:=&dlppb.TransientCryptoKey{Name:transientKeyName1,}transientCryptoKey2:=&dlppb.TransientCryptoKey{Name:transientKeyName2,}cryptoKey1:=&dlppb.CryptoKey_Transient{Transient:transientCryptoKey1,}cryptoKey2:=&dlppb.CryptoKey_Transient{Transient:transientCryptoKey2,}// Specify the CryptoHashConfig for transientKey1.cryptoHashConfig1:=&dlppb.CryptoHashConfig{CryptoKey:&dlppb.CryptoKey{Source:cryptoKey1,},}// Specify the CryptoHashConfig for transientKey1.cryptoHashConfig2:=&dlppb.CryptoHashConfig{CryptoKey:&dlppb.CryptoKey{Source:cryptoKey2,},}// Define type of de-identification as cryptographic hash transformation.// Specify the transformation for transientKey1primitiveTransformation1:=&dlppb.PrimitiveTransformation{Transformation:&dlppb.PrimitiveTransformation_CryptoHashConfig{CryptoHashConfig:cryptoHashConfig1,},}// Specify the transformation for transientKey2primitiveTransformation2:=&dlppb.PrimitiveTransformation{Transformation:&dlppb.PrimitiveTransformation_CryptoHashConfig{CryptoHashConfig:cryptoHashConfig2,},}infoTypeTransformation:=&dlppb.InfoTypeTransformations_InfoTypeTransformation{PrimitiveTransformation:primitiveTransformation2,}transformations:=&dlppb.InfoTypeTransformations{Transformations:[]*dlppb.InfoTypeTransformations_InfoTypeTransformation{infoTypeTransformation,},}// Specify fields to be de-identified.fieldIds:=[]*dlppb.FieldId{{Name:"userid"},{Name:"comments"},}fieldTransformations:=[]*dlppb.FieldTransformation{{Fields:[]*dlppb.FieldId{fieldIds[0],},Transformation:&dlppb.FieldTransformation_PrimitiveTransformation{PrimitiveTransformation:primitiveTransformation1,},},{Fields:[]*dlppb.FieldId{fieldIds[1],},Transformation:&dlppb.FieldTransformation_InfoTypeTransformations{InfoTypeTransformations:transformations,},},}recordTransformations:=&dlppb.RecordTransformations{FieldTransformations:fieldTransformations,}// Specify the config for the de-identify requestdeidentifyConfig:=&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:deidentifyConfig,InspectConfig:inspectConfig,Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"Table after de-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.CryptoHashConfig;importcom.google.privacy.dlp.v2.CryptoKey;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.InfoType;importcom.google.privacy.dlp.v2.InfoTypeTransformations;importcom.google.privacy.dlp.v2.InspectConfig;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.TransientCryptoKey;importcom.google.privacy.dlp.v2.Value;importjava.io.IOException;importjava.util.ArrayList;importjava.util.List;importjava.util.stream.Collectors;importjava.util.stream.Stream;publicclassDeIdentifyTableWithMultipleCryptoHash{publicstaticvoidmain(String[]args)throwsException{// TODO(developer): Replace these variables before running the sample.// The Google Cloud project id to use as a parent resource.StringprojectId="your-project-id";// The table to de-identify.TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("userid").build()).addHeaders(FieldId.newBuilder().setName("comments").build()).addRows(Table.Row.newBuilder().addValues(Value.newBuilder().setStringValue("user1@example.org").build()).addValues(Value.newBuilder().setStringValue("my email is user1@example.org and phone is 858-555-0222").build()).build()).addRows(Table.Row.newBuilder().addValues(Value.newBuilder().setStringValue("user2@example.org").build()).addValues(Value.newBuilder().setStringValue("my email is user2@example.org and phone is 858-555-0223").build()).build()).addRows(Table.Row.newBuilder().addValues(Value.newBuilder().setStringValue("user3@example.org").build()).addValues(Value.newBuilder().setStringValue("my email is user3@example.org and phone is 858-555-0224").build()).build()).build();// The names of the keys used to encrypt the data.StringtransientKeyName1="YOUR_TRANSIENT_CRYPTO_KEY";StringtransientKeyName2="YOUR_TRANSIENT_CRYPTO_KEY_2";deIdentifyWithCryptHashTransformation(projectId,tableToDeIdentify,transientKeyName1,transientKeyName2);}// Transforms findings using two separate cryptographic hash transformations.publicstaticvoiddeIdentifyWithCryptHashTransformation(StringprojectId,TabletableToDeIdentify,StringtransientKeyName1,StringtransientKeyName2)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to DeIdentifyContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify the type of info the inspection will look for.// See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info typesList<InfoType>infoTypes=Stream.of("PHONE_NUMBER","EMAIL_ADDRESS").map(it->InfoType.newBuilder().setName(it).build()).collect(Collectors.toList());InspectConfiginspectConfig=InspectConfig.newBuilder().addAllInfoTypes(infoTypes).build();// Specify the transient key which will encrypt the data.TransientCryptoKeytransientCryptoKey=TransientCryptoKey.newBuilder().setName(transientKeyName1).build();TransientCryptoKeytransientCryptoKey2=TransientCryptoKey.newBuilder().setName(transientKeyName2).build();CryptoKeycryptoKey=CryptoKey.newBuilder().setTransient(transientCryptoKey).build();CryptoKeycryptoKey2=CryptoKey.newBuilder().setTransient(transientCryptoKey2).build();CryptoHashConfigcryptoHashConfig=CryptoHashConfig.newBuilder().setCryptoKey(cryptoKey).build();CryptoHashConfigcryptoHashConfig2=CryptoHashConfig.newBuilder().setCryptoKey(cryptoKey2).build();// Define type of de-identification as cryptographic hash transformation.PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setCryptoHashConfig(cryptoHashConfig).build();PrimitiveTransformationprimitiveTransformation2=PrimitiveTransformation.newBuilder().setCryptoHashConfig(cryptoHashConfig2).build();InfoTypeTransformations.InfoTypeTransformationinfoTypeTransformation=InfoTypeTransformations.InfoTypeTransformation.newBuilder().setPrimitiveTransformation(primitiveTransformation2).addAllInfoTypes(infoTypes).build();InfoTypeTransformationstransformations=InfoTypeTransformations.newBuilder().addTransformations(infoTypeTransformation).build();// Specify fields to be de-identified.List<FieldId>fieldIds=Stream.of("userid").map(id->FieldId.newBuilder().setName(id).build()).collect(Collectors.toList());List<FieldId>fieldIds1=Stream.of("comments").map(id->FieldId.newBuilder().setName(id).build()).collect(Collectors.toList());List<FieldTransformation>fieldTransformations=newArrayList<>();fieldTransformations.add(FieldTransformation.newBuilder().addAllFields(fieldIds).setPrimitiveTransformation(primitiveTransformation).build());fieldTransformations.add(FieldTransformation.newBuilder().addAllFields(fieldIds1).setInfoTypeTransformations(transformations).build());RecordTransformationsrecordTransformations=RecordTransformations.newBuilder().addAllFieldTransformations(fieldTransformations).build();// Specify the config for the de-identify requestDeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(recordTransformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setInspectConfig(inspectConfig).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after de-identification: "+response.getItem().getTable());}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud client libraryconstDLP=require('@google-cloud/dlp');// Instantiates a clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// Crypto key 1// const transientKey1 = 'YOUR_TRANSIENT_CRYPTO_KEY';// Crypto key 2// const transientKey2 = 'YOUR_TRANSIENT_CRYPTO_KEY_2';// The table to de-identify.consttableToDeIdentify={headers:[{name:'userid'},{name:'comments'}],rows:[{values:[{stringValue:'user1@example.org'},{stringValue:'my email is user1@example.org and phone is 858-555-0222',},],},{values:[{stringValue:'user2@example.org'},{stringValue:'my email is user2@example.org and phone is 858-555-0223',},],},{values:[{stringValue:'user3@example.org',},{stringValue:'my email is user3@example.org and phone is 858-555-0224',},],},],};asyncfunctiondeIdentifyTableWithMultipleCryptoHash(){// The type of info the inspection will look for.constinfoTypes=[{name:'PHONE_NUMBER'},{name:'EMAIL_ADDRESS'}];// The fields to be de-identified.constfieldIds1=[{name:'userid'}];constfieldIds2=[{name:'comments'}];// Construct two primitive transformations using two different keys.constprimitiveTransformation1={cryptoHashConfig:{cryptoKey:{transient:{name:transientKey1,},},},};constprimitiveTransformation2={cryptoHashConfig:{cryptoKey:{transient:{name:transientKey2,},},},};// Construct infoType transformation using transient key 2constinfoTypeTransformation={primitiveTransformation:primitiveTransformation2,infoTypes:infoTypes,};// Associate each field with transformation defined above.constfieldTransformations=[{fields:fieldIds1,primitiveTransformation:primitiveTransformation1,},{fields:fieldIds2,infoTypeTransformations:{transformations:[infoTypeTransformation],},},];// Use transformation confiugrations and construct de-identify configuration.constdeidentifyConfig={recordTransformations:{fieldTransformations:fieldTransformations,},};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,deidentifyConfig:deidentifyConfig,inspectConfig:{infoTypes:infoTypes,},item:{table:tableToDeIdentify,},};// Send the request and receive response from the service.const[response]=awaitdlp.deidentifyContent(request);constdeidentifiedTable=response.item.table;// Print the results.console.log(`Deidentified table:${JSON.stringify(deidentifiedTable,null,2)}`);}deIdentifyTableWithMultipleCryptoHash();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\CryptoHashConfig;use Google\Cloud\Dlp\V2\CryptoKey;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\InfoType;use Google\Cloud\Dlp\V2\InfoTypeTransformations;use Google\Cloud\Dlp\V2\InfoTypeTransformations\InfoTypeTransformation;use Google\Cloud\Dlp\V2\InspectConfig;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\TransientCryptoKey;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data with multiple crypto hash. * Transform findings using two separate cryptographic hash transformations. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify. * @param string $outputCsvFile The oupt file path to save deidentify content. * @param string $transientCryptoKeyName1 Specify the random string. * @param string $transientCryptoKeyName2 Specify the random string. */function deidentify_table_with_multiple_crypto_hash( // TODO(developer): Replace sample parameters before running the code. string $callingProjectId, string $inputCsvFile = './test/data/table6.csv', string $outputCsvFile = './test/data/deidentify_table_with_multiple_crypto_hash_output.csv', string $transientCryptoKeyName1 = 'YOUR-TRANSIENT-CRYPTO-KEY-1', string $transientCryptoKeyName2 = 'YOUR-TRANSIENT-CRYPTO-KEY-2'): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file. $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects. $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object. $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify what content you want the service to de-identify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify the type of info the inspection will look for. // See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info types $infoTypes = [ (new InfoType())->setName('EMAIL_ADDRESS'), (new InfoType())->setName('PHONE_NUMBER') ]; $inspectConfig = (new InspectConfig()) ->setInfoTypes($infoTypes); // ---- First Crypto Hash Rule ---- // Specify the transient key which will encrypt the data. $cryptoHashConfig1 = (new CryptoHashConfig()) ->setCryptoKey((new CryptoKey()) ->setTransient((new TransientCryptoKey()) ->setName($transientCryptoKeyName1))); // Define type of de-identification as cryptographic hash transformation. $primitiveTransformation1 = (new PrimitiveTransformation()) ->setCryptoHashConfig($cryptoHashConfig1); $fieldTransformation1 = (new FieldTransformation()) ->setPrimitiveTransformation($primitiveTransformation1) // Specify fields to be de-identified. ->setFields([ (new FieldId())->setName('userid') ]); // ---- Second Crypto Hash Rule ---- // Specify the transient key which will encrypt the data. $cryptoHashConfig2 = (new CryptoHashConfig()) ->setCryptoKey((new CryptoKey()) ->setTransient((new TransientCryptoKey()) ->setName($transientCryptoKeyName2))); // Define type of de-identification as cryptographic hash transformation. $primitiveTransformation2 = (new PrimitiveTransformation()) ->setCryptoHashConfig($cryptoHashConfig2); $infoTypeTransformation = (new InfoTypeTransformation()) ->setPrimitiveTransformation($primitiveTransformation2) ->setInfoTypes($infoTypes); $infoTypeTransformations = (new InfoTypeTransformations()) ->setTransformations([$infoTypeTransformation]); $fieldTransformation2 = (new FieldTransformation()) ->setInfoTypeTransformations($infoTypeTransformations) // Specify fields to be de-identified. ->setFields([ (new FieldId())->setName('comments') ]); $recordtransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation1, $fieldTransformation2]); // Specify the config for the de-identify request $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Send the request and receive response from the service. $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setInspectConfig($inspectConfig) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print the results. $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf('Table after deidentify (File Location): %s', $outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportDict,List,Unionimportgoogle.cloud.dlpdefdeidentify_table_with_multiple_crypto_hash(project:str,table_data:Dict[str,Union[List[str],List[List[str]]]],info_types:List[str],transient_key_name_1:str,transient_key_name_2:str,deid_fields_1:List[str],deid_fields_2:List[str],)->None:"""Uses the Data Loss Prevention API to de-identify sensitive data in table using multiple transient cryptographic hash keys. Args: project: The Google Cloud project id to use as a parent resource. table_data: Dictionary representing table data. info_types: A list of strings representing info types to look for. A full list of info type categories can be fetched from the API. transient_key_name_1: Name of the first transient crypto key used for encryption. The scope of this key is a single API call. It is generated for the transformation and then discarded. transient_key_name_2: Name of the second transient crypto key used for encryption. The scope of this key is a single API call. It is generated for the transformation and then discarded. deid_fields_1: List of column names in table to de-identify using transient_key_name_1. deid_fields_2: List of column names in table to de-identify using transient_key_name_2. """# Instantiate a clientdlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_data["header"]]rows=[]forrowintable_data["rows"]:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the `item`item={"table":table}# Prepare info_types by converting the list of strings into a list of# dictionaries.info_types=[{"name":info_type}forinfo_typeininfo_types]# Construct cryptographic hash configurations using two transient keys# which will encrypt the data.crypto_hash_config_1={"crypto_key":{"transient":{"name":transient_key_name_1}}}crypto_hash_config_2={"crypto_key":{"transient":{"name":transient_key_name_2}}}# Prepare fields to be de-identified by converting list of strings# into list of dictionaries.deid_fields_1=[{"name":field}forfieldindeid_fields_1]deid_fields_2=[{"name":field}forfieldindeid_fields_2]# Specify the type of info the inspection will look for.inspect_config={"info_types":info_types,}# Construct deidentify configuration dictionary.deidentify_config={"record_transformations":{"field_transformations":[{"fields":deid_fields_1,"primitive_transformation":{"crypto_hash_config":crypto_hash_config_1},},{"fields":deid_fields_2,"info_type_transformations":{"transformations":[{"info_types":info_types,"primitive_transformation":{"crypto_hash_config":crypto_hash_config_2},}]},},]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"inspect_config":inspect_config,"item":item,})# Print the result.print(f"Table after de-identification:{response.item.table}")REST
{ "inspectConfig":{ "infoTypes":[ { "name":"EMAIL_ADDRESS" }, { "name":"PHONE_NUMBER" } ] }, "deidentifyConfig":{ "recordTransformations":{ "fieldTransformations":[ { "fields":[ { "name":"userid" } ], "primitiveTransformation":{ "cryptoHashConfig":{ "cryptoKey":{ "transient":{ "name":"[TRANSIENT-CRYPTO-KEY-1]" } } } } }, { "fields":[ { "name":"comments" } ], "infoTypeTransformations":{ "transformations":[ { "infoTypes":[ { "name":"PHONE_NUMBER" }, { "name":"EMAIL_ADDRESS" } ], "primitiveTransformation":{ "cryptoHashConfig":{ "cryptoKey":{ "transient":{ "name":"[TRANSIENT-CRYPTO-KEY-2]" } } } } } ] } } ] } }, "item":{ "table":{ "headers":[ { "name":"userid" }, { "name":"comments" } ], "rows":[ { "values":[ { "stringValue":"user1@example.org" }, { "stringValue":"my email is user1@example.org and phone is 858-333-2222" } ] }, { "values":[ { "stringValue":"abbyabernathy1" }, { "stringValue":"my userid is abbyabernathy1 and my email is aabernathy@example.com" } ] } ] } } }Transform findings using format-preserving encryption
This example demonstrates how to useSensitive Data Protection to replace a sensitive string with a token that has thesame length and alphabet as the original string. This example uses theCryptoReplaceFfxFpeConfigtransformation method. For more information, seeFormat-preservingencryption.
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;usingGoogle.Protobuf;usingstaticGoogle.Cloud.Dlp.V2.CryptoReplaceFfxFpeConfig.Types;publicclassDeidentifyTableWithFpe{publicstaticTableDeidentifyTable(stringprojectId,stringkeyName,stringwrappedKey,FfxCommonNativeAlphabetalphabet=FfxCommonNativeAlphabet.Numeric,TabletableToInspect=null){// Instantiate a client.vardlp=DlpServiceClient.Create();// Construct the table if null.if(tableToInspect==null){varrow1=newValue[]{newValue{StringValue="11111"},newValue{StringValue="2015"},newValue{StringValue="$10"}};varrow2=newValue[]{newValue{StringValue="33333"},newValue{StringValue="2016"},newValue{StringValue="$20"}};varrow3=newValue[]{newValue{StringValue="22222"},newValue{StringValue="2016"},newValue{StringValue="$15"}};tableToInspect=newTable{Headers={newFieldId{Name="Employee ID"},newFieldId{Name="Date"},newFieldId{Name="Compensation"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}},newTable.Types.Row{Values={row3}}}};}// Provide the table and construct the content item.varcontentItem=newContentItem{Table=tableToInspect};// Specify an encrypted AES-256 key and the name of the Cloud KMS Key that// encrypted it and specify how it should be encrypted.varcryptoReplaceFfxFpeConfig=newCryptoReplaceFfxFpeConfig{CryptoKey=newCryptoKey{KmsWrapped=newKmsWrappedCryptoKey{CryptoKeyName=keyName,WrappedKey=ByteString.FromBase64(wrappedKey)}},CommonAlphabet=alphabet};// Specify fields to be encrypted.varfields=newFieldId[]{newFieldId{Name="Employee ID"}};// Construct the deidentify config using crypto replace config created above.vardeidentifyConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={newFieldTransformation{PrimitiveTransformation=newPrimitiveTransformation{CryptoReplaceFfxFpeConfig=cryptoReplaceFfxFpeConfig},Fields={fields}}}}};// Construct the request.varrequest=newDeidentifyContentRequest{ParentAsLocationName=newLocationName(projectId,"global"),DeidentifyConfig=deidentifyConfig,Item=contentItem,};// Call the API.DeidentifyContentResponseresponse=dlp.DeidentifyContent(request);// Inspect the response.Console.WriteLine(response.Item.Table);returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""encoding/base64""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// deidentifyTableFPE de-identifies table data with format preserving encryption.funcdeidentifyTableFPE(wio.Writer,projectIDstring,kmsKeyName,wrappedAESKeystring)error{// projectId := "your-project-id"/* keyFileName := "projects/YOUR_PROJECT/" + "locations/YOUR_KEYRING_REGION/" + "keyRings/YOUR_KEYRING_NAME/" + "cryptoKeys/YOUR_KEY_NAME"*/// wrappedAESKey := "YOUR_ENCRYPTED_AES_256_KEY"// define your table.row1:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"11111"}},{Type:&dlppb.Value_StringValue{StringValue:"2015"}},{Type:&dlppb.Value_StringValue{StringValue:"$10"}},},}row2:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"22222"}},{Type:&dlppb.Value_StringValue{StringValue:"2016"}},{Type:&dlppb.Value_StringValue{StringValue:"$20"}},},}row3:=&dlppb.Table_Row{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"33333"}},{Type:&dlppb.Value_StringValue{StringValue:"2016"}},{Type:&dlppb.Value_StringValue{StringValue:"$15"}},},}table:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"Employee ID"},{Name:"Date"},{Name:"Compensation"},},Rows:[]*dlppb.Table_Row{{Values:row1.Values},{Values:row2.Values},{Values:row3.Values},},}ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify what content you want the service to de-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:table,},}// Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted it.kmsKeyDecode,err:=base64.StdEncoding.DecodeString(wrappedAESKey)iferr!=nil{returnfmt.Errorf("error in decoding key: %w",err)}kmsWrappedCryptoKey:=&dlppb.KmsWrappedCryptoKey{WrappedKey:kmsKeyDecode,CryptoKeyName:kmsKeyName,}cryptoKey:=&dlppb.CryptoKey_KmsWrapped{KmsWrapped:kmsWrappedCryptoKey,}// Specify how the content should be encrypted.cryptoReplaceFfxFpeConfig:=&dlppb.CryptoReplaceFfxFpeConfig{CryptoKey:&dlppb.CryptoKey{Source:cryptoKey,},Alphabet:&dlppb.CryptoReplaceFfxFpeConfig_CommonAlphabet{CommonAlphabet:dlppb.CryptoReplaceFfxFpeConfig_NUMERIC,},}primitiveTransformation:=&dlppb.PrimitiveTransformation{Transformation:&dlppb.PrimitiveTransformation_CryptoReplaceFfxFpeConfig{CryptoReplaceFfxFpeConfig:cryptoReplaceFfxFpeConfig,},}// Specify field to be encrypted.fieldId:=&dlppb.FieldId{Name:"Employee ID",}// Associate the encryption with the specified field.fieldTransformation:=&dlppb.FieldTransformation{Fields:[]*dlppb.FieldId{fieldId,},Transformation:&dlppb.FieldTransformation_PrimitiveTransformation{PrimitiveTransformation:primitiveTransformation,},}transformations:=&dlppb.RecordTransformations{FieldTransformations:[]*dlppb.FieldTransformation{fieldTransformation,},}// Construct the de-identification request to be sent by the client.req:=&dlppb.DeidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),DeidentifyConfig:&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:transformations,},},Item:contentItem,}// Send the request.resp,err:=client.DeidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the results.fmt.Fprintf(w,"De-identify Table after format-preserving encryption : %+v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.common.io.BaseEncoding;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.CryptoKey;importcom.google.privacy.dlp.v2.CryptoReplaceFfxFpeConfig;importcom.google.privacy.dlp.v2.CryptoReplaceFfxFpeConfig.FfxCommonNativeAlphabet;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.DeidentifyContentRequest;importcom.google.privacy.dlp.v2.DeidentifyContentResponse;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.KmsWrappedCryptoKey;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importcom.google.protobuf.ByteString;importjava.io.IOException;publicclassDeIdentifyTableWithFpe{publicstaticvoidmain(String[]args)throwsException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";StringkmsKeyName="projects/YOUR_PROJECT/"+"locations/YOUR_KEYRING_REGION/"+"keyRings/YOUR_KEYRING_NAME/"+"cryptoKeys/YOUR_KEY_NAME";StringwrappedAesKey="YOUR_ENCRYPTED_AES_256_KEY";TabletableToDeIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("Employee ID").build()).addHeaders(FieldId.newBuilder().setName("Date").build()).addHeaders(FieldId.newBuilder().setName("Compensation").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("11111").build()).addValues(Value.newBuilder().setStringValue("2015").build()).addValues(Value.newBuilder().setStringValue("$10").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("22222").build()).addValues(Value.newBuilder().setStringValue("2016").build()).addValues(Value.newBuilder().setStringValue("$20").build()).build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("33333").build()).addValues(Value.newBuilder().setStringValue("2016").build()).addValues(Value.newBuilder().setStringValue("$15").build()).build()).build();deIdentifyTableWithFpe(projectId,tableToDeIdentify,kmsKeyName,wrappedAesKey);}publicstaticvoiddeIdentifyTableWithFpe(StringprojectId,TabletableToDeIdentify,StringkmsKeyName,StringwrappedAesKey)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to de-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToDeIdentify).build();// Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted itKmsWrappedCryptoKeykmsWrappedCryptoKey=KmsWrappedCryptoKey.newBuilder().setWrappedKey(ByteString.copyFrom(BaseEncoding.base64().decode(wrappedAesKey))).setCryptoKeyName(kmsKeyName).build();CryptoKeycryptoKey=CryptoKey.newBuilder().setKmsWrapped(kmsWrappedCryptoKey).build();// Specify how the content should be encrypted.CryptoReplaceFfxFpeConfigcryptoReplaceFfxFpeConfig=CryptoReplaceFfxFpeConfig.newBuilder().setCryptoKey(cryptoKey)// Set of characters in the input text. For more info, see// https://cloud.google.com/dlp/docs/reference/rest/v2/organizations.deidentifyTemplates#DeidentifyTemplate.FfxCommonNativeAlphabet.setCommonAlphabet(FfxCommonNativeAlphabet.NUMERIC).build();PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setCryptoReplaceFfxFpeConfig(cryptoReplaceFfxFpeConfig).build();// Specify field to be encrypted.FieldIdfieldId=FieldId.newBuilder().setName("Employee ID").build();// Associate the encryption with the specified field.FieldTransformationfieldTransformation=FieldTransformation.newBuilder().setPrimitiveTransformation(primitiveTransformation).addFields(fieldId).build();RecordTransformationstransformations=RecordTransformations.newBuilder().addFieldTransformations(fieldTransformation).build();DeidentifyConfigdeidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.DeidentifyContentRequestrequest=DeidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setDeidentifyConfig(deidentifyConfig).build();// Send the request and receive response from the service.DeidentifyContentResponseresponse=dlp.deidentifyContent(request);// Print the results.System.out.println("Table after format-preserving encryption: "+response.getItem().getTable());}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Instantiates a clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// The set of characters to replace sensitive ones with// For more information, see https://cloud.google.com/dlp/docs/reference/rest/v2/organizations.deidentifyTemplates#ffxcommonnativealphabet// const alphabet = 'NUMERIC';// The name of the Cloud KMS key used to encrypt ('wrap') the AES-256 key// const keyName = 'projects/YOUR_GCLOUD_PROJECT/locations/YOUR_LOCATION/keyRings/YOUR_KEYRING_NAME/cryptoKeys/YOUR_KEY_NAME';// The encrypted ('wrapped') AES-256 key to use// This key should be encrypted using the Cloud KMS key specified above// const wrappedKey = 'YOUR_ENCRYPTED_AES_256_KEY'// Table to de-identifyconsttablularData={headers:[{name:'Employee ID'},{name:'Date'},{name:'Compensation'}],rows:[{values:[{stringValue:'11111'},{stringValue:'2015'},{stringValue:'$10'},],},{values:[{stringValue:'22222'},{stringValue:'2016'},{stringValue:'$20'},],},{values:[{stringValue:'33333'},{stringValue:'2016'},{stringValue:'$15'},],},],};asyncfunctiondeidentifyTableWithFpe(){// Specify field to be encrypted.constfieldIds=[{name:'Employee ID'}];// Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted itconstcryptoKeyConfig={kmsWrapped:{wrappedKey:wrappedKey,cryptoKeyName:keyName,},};// Specify how the content should be encrypted.constcryptoReplaceFfxFpeConfig={cryptoKey:cryptoKeyConfig,commonAlphabet:alphabet,};// Associate the encryption with the specified field.constfieldTransformations=[{fields:fieldIds,primitiveTransformation:{cryptoReplaceFfxFpeConfig,},},];// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,deidentifyConfig:{recordTransformations:{fieldTransformations,},},item:{table:tablularData,},};// Send the request and receive response from the service.const[response]=awaitdlp.deidentifyContent(request);// Print the results.console.log(`Table after de-identification:${JSON.stringify(response.item.table)}`);}awaitdeidentifyTableWithFpe();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\CryptoKey;use Google\Cloud\Dlp\V2\CryptoReplaceFfxFpeConfig;use Google\Cloud\Dlp\V2\CryptoReplaceFfxFpeConfig\FfxCommonNativeAlphabet;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\DeidentifyContentRequest;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\KmsWrappedCryptoKey;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * De-identify table data with format-preserving encryption. * Demonstrates encrypting sensitive data in a table while maintaining format. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to deidentify. * @param string $outputCsvFile The oupt file path to save deidentify content. * @param string $encryptedFieldNames The field to be encrypted. * @param string $kmsKeyName The name of the Cloud KMS key used to encrypt ('wrap') the AES-256 key. * Example: key_name = 'projects/YOUR_GCLOUD_PROJECT/locations/YOUR_LOCATION/keyRings/YOUR_KEYRING_NAME/cryptoKeys/YOUR_KEY_NAME' * @param string $wrappedAesKey The encrypted ('wrapped') AES-256 key to use. * */function deidentify_table_fpe( string $callingProjectId, string $inputCsvFile, string $outputCsvFile, string $encryptedFieldNames, string $kmsKeyName, string $wrappedAesKey): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file. $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects. $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object. $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify the content to be de-identify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted it. $kmsWrappedCryptoKey = (new KmsWrappedCryptoKey()) ->setWrappedKey(base64_decode($wrappedAesKey)) ->setCryptoKeyName($kmsKeyName); $cryptoKey = (new CryptoKey()) ->setKmsWrapped($kmsWrappedCryptoKey); // Specify how the content should be encrypted. $cryptoReplaceFfxFpeConfig = (new CryptoReplaceFfxFpeConfig()) ->setCryptoKey($cryptoKey) ->setCommonAlphabet(FfxCommonNativeAlphabet::NUMERIC); $primitiveTransformation = (new PrimitiveTransformation()) ->setCryptoReplaceFfxFpeConfig($cryptoReplaceFfxFpeConfig); // Specify field to be encrypted. $encryptedFields = array_map(function ($encryptedFieldName) { return (new FieldId()) ->setName($encryptedFieldName); }, explode(',', $encryptedFieldNames)); // Associate the encryption with the specified field. $fieldTransformation = (new FieldTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setFields($encryptedFields); $recordtransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation]); $deidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Run request. $deidentifyContentRequest = (new DeidentifyContentRequest()) ->setParent($parent) ->setDeidentifyConfig($deidentifyConfig) ->setItem($content); $response = $dlp->deidentifyContent($deidentifyContentRequest); // Print the results. $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf('Table after format-preserving encryption (File Location): %s', $outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
fromtypingimportListimportgoogle.cloud.dlpdefdeidentify_table_with_fpe(project:str,table_header:List[str],table_rows:List[List[str]],deid_field_names:List[str],key_name:str=None,wrapped_key:bytes=None,alphabet:str=None,)->None:"""Uses the Data Loss Prevention API to de-identify sensitive data in a table while maintaining format. Args: project: The Google Cloud project id to use as a parent resource. table_header: List of strings representing table field names. table_rows: List of rows representing table data. deid_field_names: A list of fields in table to de-identify. key_name: The name of the Cloud KMS key used to encrypt ('wrap') the AES-256 key. Example: key_name = 'projects/YOUR_GCLOUD_PROJECT/locations/YOUR_LOCATION/ keyRings/YOUR_KEYRING_NAME/cryptoKeys/YOUR_KEY_NAME' wrapped_key: The decrypted ('wrapped', in bytes) AES-256 key to use. This key should be encrypted using the Cloud KMS key specified by key_name. alphabet: The set of characters to replace sensitive ones with. For more information, see https://cloud.google.com/dlp/docs/reference/ rest/v2/projects.deidentifyTemplates#ffxcommonnativealphabet """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_header]rows=[]forrowintable_rows:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Construct the `item` for table.item={"table":table}# Specify fields to be de-identified.deid_field_names=[{"name":_i}for_iindeid_field_names]# Construct FPE configuration dictionarycrypto_replace_ffx_fpe_config={"crypto_key":{"kms_wrapped":{"wrapped_key":wrapped_key,"crypto_key_name":key_name},},"common_alphabet":alphabet,}# Construct deidentify configuration dictionarydeidentify_config={"record_transformations":{"field_transformations":[{"primitive_transformation":{"crypto_replace_ffx_fpe_config":crypto_replace_ffx_fpe_config},"fields":deid_field_names,}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.deidentify_content(request={"parent":parent,"deidentify_config":deidentify_config,"item":item})# Print out results.print(f"Table after de-identification:{response.item.table}")Re-identify findings de-identified using format-preserving encryption
This example demonstrates how to useSensitive Data Protection to re-identify sensitive data in tables that werede-identified through theCryptoReplaceFfxFpeConfigtransformation method. For more information, seeFormat-preservingencryption.
C#
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
usingSystem;usingGoogle.Api.Gax.ResourceNames;usingGoogle.Cloud.Dlp.V2;usingGoogle.Protobuf;usingstaticGoogle.Cloud.Dlp.V2.CryptoReplaceFfxFpeConfig.Types;publicclassReidentifyTableDataWithFpe{publicstaticTableReidentifyTableData(stringprojectId,stringkeyName,stringwrappedKey,FfxCommonNativeAlphabetalphabet=FfxCommonNativeAlphabet.Numeric,TabletableToInspect=null){// Instantiate a client.vardlp=DlpServiceClient.Create();// Construct the table if null.vartable=tableToInspect;if(table==null){varrow1=newValue[]{newValue{StringValue="28777"},newValue{StringValue="Justin"}};varrow2=newValue[]{newValue{StringValue="28778"},newValue{StringValue="Gary"}};table=newTable{Headers={newFieldId{Name="Employee ID"},newFieldId{Name="Employee Name"}},Rows={newTable.Types.Row{Values={row1}},newTable.Types.Row{Values={row2}}}};}// Construct the content item by providing the table.varcontentItem=newContentItem{Table=table};// Specify how to decrypt the previously de-identified information.varcryptoReplaceFfxFpeConfig=newCryptoReplaceFfxFpeConfig{CryptoKey=newCryptoKey{KmsWrapped=newKmsWrappedCryptoKey{CryptoKeyName=keyName,WrappedKey=ByteString.FromBase64(wrappedKey)}},CommonAlphabet=alphabet};// Specify the field to be decrypted.varfields=newFieldId[]{newFieldId{Name="Employee ID"}};// Construct the re-identify config and specify the transformation.varreidentifyConfig=newDeidentifyConfig{RecordTransformations=newRecordTransformations{FieldTransformations={newFieldTransformation{PrimitiveTransformation=newPrimitiveTransformation{CryptoReplaceFfxFpeConfig=cryptoReplaceFfxFpeConfig,},Fields={fields}}}}};// Construct the request.varrequest=newReidentifyContentRequest{Parent=newLocationName(projectId,"global").ToString(),Item=contentItem,ReidentifyConfig=reidentifyConfig};// Call the API.ReidentifyContentResponseresponse=dlp.ReidentifyContent(request);// Inspect the response.Console.WriteLine($"Table after re-identification: {response.Item.Table}");returnresponse.Item.Table;}}Go
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
import("context""encoding/base64""fmt""io"dlp"cloud.google.com/go/dlp/apiv2""cloud.google.com/go/dlp/apiv2/dlppb")// reidTableDataWithFPE re-identifies table data with FPEfuncreidTableDataWithFPE(wio.Writer,projectID,kmsKeyName,wrappedAesKeystring)error{// projectId := "my-project-id"/* kmsKeyName := "projects/YOUR_PROJECT/" + "locations/YOUR_KEYRING_REGION/" + "keyRings/YOUR_KEYRING_NAME/" + "cryptoKeys/YOUR_KEY_NAME"*/// wrappedAesKey := "YOUR_ENCRYPTED_AES_256_KEY"ctx:=context.Background()// Initialize a client once and reuse it to send multiple requests. Clients// are safe to use across goroutines. When the client is no longer needed,// call the Close method to cleanup its resources.client,err:=dlp.NewClient(ctx)iferr!=nil{returnerr}// Closing the client safely cleans up background resources.deferclient.Close()// Specify the table data that needs to be re-identified.tableToReIdentify:=&dlppb.Table{Headers:[]*dlppb.FieldId{{Name:"Employee ID"},},Rows:[]*dlppb.Table_Row{{Values:[]*dlppb.Value{{Type:&dlppb.Value_StringValue{StringValue:"90511",},},},},},}// Specify the content you want to re-identify.contentItem:=&dlppb.ContentItem{DataItem:&dlppb.ContentItem_Table{Table:tableToReIdentify,},}// Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted it.kmsWrappedCryptoKey,err:=base64.StdEncoding.DecodeString(wrappedAesKey)iferr!=nil{returnerr}cryptoKey:=&dlppb.CryptoKey{Source:&dlppb.CryptoKey_KmsWrapped{KmsWrapped:&dlppb.KmsWrappedCryptoKey{WrappedKey:kmsWrappedCryptoKey,CryptoKeyName:kmsKeyName,},},}// Specify how to un-encrypt the previously de-identified information.cryptoReplaceFfxFpeConfig:=&dlppb.CryptoReplaceFfxFpeConfig{CryptoKey:cryptoKey,Alphabet:&dlppb.CryptoReplaceFfxFpeConfig_CommonAlphabet{CommonAlphabet:dlppb.CryptoReplaceFfxFpeConfig_NUMERIC,},}primitiveTransformation:=&dlppb.PrimitiveTransformation{Transformation:&dlppb.PrimitiveTransformation_CryptoReplaceFfxFpeConfig{CryptoReplaceFfxFpeConfig:cryptoReplaceFfxFpeConfig,},}// Specify field to be decrypted.fieldId:=&dlppb.FieldId{Name:"Employee ID",}// Associate the decryption with the specified field.fieldTransformation:=&dlppb.FieldTransformation{Transformation:&dlppb.FieldTransformation_PrimitiveTransformation{PrimitiveTransformation:primitiveTransformation,},Fields:[]*dlppb.FieldId{fieldId,},}recordTransformations:=&dlppb.RecordTransformations{FieldTransformations:[]*dlppb.FieldTransformation{fieldTransformation,},}deidentifyConfig:=&dlppb.DeidentifyConfig{Transformation:&dlppb.DeidentifyConfig_RecordTransformations{RecordTransformations:recordTransformations,},}// Combine configurations into a request for the service.req:=&dlppb.ReidentifyContentRequest{Parent:fmt.Sprintf("projects/%s/locations/global",projectID),ReidentifyConfig:deidentifyConfig,Item:contentItem,}// Send the request and receive response from the service.resp,err:=client.ReidentifyContent(ctx,req)iferr!=nil{returnerr}// Print the result.fmt.Fprintf(w,"Table after re-identification : %v",resp.GetItem().GetTable())returnnil}Java
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importcom.google.cloud.dlp.v2.DlpServiceClient;importcom.google.common.io.BaseEncoding;importcom.google.privacy.dlp.v2.ContentItem;importcom.google.privacy.dlp.v2.CryptoKey;importcom.google.privacy.dlp.v2.CryptoReplaceFfxFpeConfig;importcom.google.privacy.dlp.v2.CryptoReplaceFfxFpeConfig.FfxCommonNativeAlphabet;importcom.google.privacy.dlp.v2.DeidentifyConfig;importcom.google.privacy.dlp.v2.FieldId;importcom.google.privacy.dlp.v2.FieldTransformation;importcom.google.privacy.dlp.v2.KmsWrappedCryptoKey;importcom.google.privacy.dlp.v2.LocationName;importcom.google.privacy.dlp.v2.PrimitiveTransformation;importcom.google.privacy.dlp.v2.RecordTransformations;importcom.google.privacy.dlp.v2.ReidentifyContentRequest;importcom.google.privacy.dlp.v2.ReidentifyContentResponse;importcom.google.privacy.dlp.v2.Table;importcom.google.privacy.dlp.v2.Table.Row;importcom.google.privacy.dlp.v2.Value;importcom.google.protobuf.ByteString;importjava.io.IOException;publicclassReIdentifyTableWithFpe{publicstaticvoidmain(String[]args)throwsException{// TODO(developer): Replace these variables before running the sample.StringprojectId="your-project-id";StringkmsKeyName="projects/YOUR_PROJECT/"+"locations/YOUR_KEYRING_REGION/"+"keyRings/YOUR_KEYRING_NAME/"+"cryptoKeys/YOUR_KEY_NAME";StringwrappedAesKey="YOUR_ENCRYPTED_AES_256_KEY";TabletableToReIdentify=Table.newBuilder().addHeaders(FieldId.newBuilder().setName("Employee ID").build()).addRows(Row.newBuilder().addValues(Value.newBuilder().setStringValue("28777").build()).build()).build();reIdentifyTableWithFpe(projectId,tableToReIdentify,kmsKeyName,wrappedAesKey);}publicstaticvoidreIdentifyTableWithFpe(StringprojectId,TabletableToReIdentify,StringkmsKeyName,StringwrappedAesKey)throwsIOException{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests. After completing all of your requests, call// the "close" method on the client to safely clean up any remaining background resources.try(DlpServiceClientdlp=DlpServiceClient.create()){// Specify what content you want the service to re-identify.ContentItemcontentItem=ContentItem.newBuilder().setTable(tableToReIdentify).build();// Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted it.KmsWrappedCryptoKeykmsWrappedCryptoKey=KmsWrappedCryptoKey.newBuilder().setWrappedKey(ByteString.copyFrom(BaseEncoding.base64().decode(wrappedAesKey))).setCryptoKeyName(kmsKeyName).build();CryptoKeycryptoKey=CryptoKey.newBuilder().setKmsWrapped(kmsWrappedCryptoKey).build();// Specify how to un-encrypt the previously de-identified information.CryptoReplaceFfxFpeConfigcryptoReplaceFfxFpeConfig=CryptoReplaceFfxFpeConfig.newBuilder().setCryptoKey(cryptoKey)// Set of characters in the input text. For more info, see// https://cloud.google.com/dlp/docs/reference/rest/v2/organizations.deidentifyTemplates#DeidentifyTemplate.FfxCommonNativeAlphabet.setCommonAlphabet(FfxCommonNativeAlphabet.NUMERIC).build();PrimitiveTransformationprimitiveTransformation=PrimitiveTransformation.newBuilder().setCryptoReplaceFfxFpeConfig(cryptoReplaceFfxFpeConfig).build();// Specify field to be decrypted.FieldIdfieldId=FieldId.newBuilder().setName("Employee ID").build();// Associate the decryption with the specified field.FieldTransformationfieldTransformation=FieldTransformation.newBuilder().setPrimitiveTransformation(primitiveTransformation).addFields(fieldId).build();RecordTransformationstransformations=RecordTransformations.newBuilder().addFieldTransformations(fieldTransformation).build();DeidentifyConfigreidentifyConfig=DeidentifyConfig.newBuilder().setRecordTransformations(transformations).build();// Combine configurations into a request for the service.ReidentifyContentRequestrequest=ReidentifyContentRequest.newBuilder().setParent(LocationName.of(projectId,"global").toString()).setItem(contentItem).setReidentifyConfig(reidentifyConfig).build();// Send the request and receive response from the serviceReidentifyContentResponseresponse=dlp.reidentifyContent(request);// Print the resultsSystem.out.println("Table after re-identification: "+response.getItem().getValue());}}}Node.js
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
// Imports the Google Cloud Data Loss Prevention libraryconstDLP=require('@google-cloud/dlp');// Instantiates a clientconstdlp=newDLP.DlpServiceClient();// The project ID to run the API call under// const projectId = 'my-project';// The set of characters to replace sensitive ones with// For more information, see https://cloud.google.com/dlp/docs/reference/rest/v2/organizations.deidentifyTemplates#ffxcommonnativealphabet// const alphabet = 'ALPHA_NUMERIC';// The name of the Cloud KMS key used to encrypt ('wrap') the AES-256 key// const keyName = 'projects/YOUR_GCLOUD_PROJECT/locations/YOUR_LOCATION/keyRings/YOUR_KEYRING_NAME/cryptoKeys/YOUR_KEY_NAME';// The encrypted ('wrapped') AES-256 key to use// This key should be encrypted using the Cloud KMS key specified above// const wrappedKey = 'YOUR_ENCRYPTED_AES_256_KEY'// The name of the surrogate custom info type to use when reidentifying data// const surrogateType = 'SOME_INFO_TYPE_DEID';// Table to re-identifyconsttablularData={headers:[{name:'Employee ID'}],rows:[{values:[{stringValue:'90511'}]}],};asyncfunctionreidentifyTableWithFpe(){// Specify field to be re-identified.constfieldIds=[{name:'Employee ID'}];// Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted itconstcryptoKeyConfig={kmsWrapped:{wrappedKey:wrappedKey,cryptoKeyName:keyName,},};// Associate transformation with crypto key congurations.constprimitiveTransformation={cryptoReplaceFfxFpeConfig:{cryptoKey:cryptoKeyConfig,commonAlphabet:alphabet,},};// Combine configurations into a request for the service.constrequest={parent:`projects/${projectId}/locations/global`,reidentifyConfig:{recordTransformations:{fieldTransformations:[{fields:fieldIds,primitiveTransformation:primitiveTransformation,},],},},item:{table:tablularData,},};// Send the request and receive response from the service.const[response]=awaitdlp.reidentifyContent(request);// Print the results.console.log(`Table after re-identification:${JSON.stringify(response.item.table)}`);}awaitreidentifyTableWithFpe();PHP
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;use Google\Cloud\Dlp\V2\ContentItem;use Google\Cloud\Dlp\V2\CryptoKey;use Google\Cloud\Dlp\V2\CryptoReplaceFfxFpeConfig;use Google\Cloud\Dlp\V2\CryptoReplaceFfxFpeConfig\FfxCommonNativeAlphabet;use Google\Cloud\Dlp\V2\DeidentifyConfig;use Google\Cloud\Dlp\V2\FieldId;use Google\Cloud\Dlp\V2\FieldTransformation;use Google\Cloud\Dlp\V2\KmsWrappedCryptoKey;use Google\Cloud\Dlp\V2\PrimitiveTransformation;use Google\Cloud\Dlp\V2\RecordTransformations;use Google\Cloud\Dlp\V2\ReidentifyContentRequest;use Google\Cloud\Dlp\V2\Table;use Google\Cloud\Dlp\V2\Table\Row;use Google\Cloud\Dlp\V2\Value;/** * Re-identify table data with FPE. * * @param string $callingProjectId The Google Cloud project id to use as a parent resource. * @param string $inputCsvFile The input file(csv) path to reidentify. * @param string $outputCsvFile The oupt file path to save reidentify content. * @param string $encryptedFieldNames The field to be encrypted. * @param string $kmsKeyName The name of the Cloud KMS key used to encrypt ('wrap') the AES-256 key. * Example: key_name = 'projects/YOUR_GCLOUD_PROJECT/locations/YOUR_LOCATION/keyRings/YOUR_KEYRING_NAME/cryptoKeys/YOUR_KEY_NAME' * @param string $wrappedAesKey The encrypted ('wrapped') AES-256 key to use. * * */function reidentify_table_fpe( string $callingProjectId, string $inputCsvFile, string $outputCsvFile, string $encryptedFieldNames, string $kmsKeyName, string $wrappedAesKey): void { // Instantiate a client. $dlp = new DlpServiceClient(); $parent = "projects/$callingProjectId/locations/global"; // Read a CSV file. $csvLines = file($inputCsvFile, FILE_IGNORE_NEW_LINES); $csvHeaders = explode(',', $csvLines[0]); $csvRows = array_slice($csvLines, 1); // Convert CSV file into protobuf objects. $tableHeaders = array_map(function ($csvHeader) { return (new FieldId) ->setName($csvHeader); }, $csvHeaders); $tableRows = array_map(function ($csvRow) { $rowValues = array_map(function ($csvValue) { return (new Value()) ->setStringValue($csvValue); }, explode(',', $csvRow)); return (new Row()) ->setValues($rowValues); }, $csvRows); // Construct the table object. $tableToDeIdentify = (new Table()) ->setHeaders($tableHeaders) ->setRows($tableRows); // Specify the content to be reidentify. $content = (new ContentItem()) ->setTable($tableToDeIdentify); // Specify an encrypted AES-256 key and the name of the Cloud KMS key that encrypted it. $kmsWrappedCryptoKey = (new KmsWrappedCryptoKey()) ->setWrappedKey(base64_decode($wrappedAesKey)) ->setCryptoKeyName($kmsKeyName); $cryptoKey = (new CryptoKey()) ->setKmsWrapped($kmsWrappedCryptoKey); // Specify how to un-encrypt the previously de-identified information. $cryptoReplaceFfxFpeConfig = (new CryptoReplaceFfxFpeConfig()) ->setCryptoKey($cryptoKey) ->setCommonAlphabet(FfxCommonNativeAlphabet::NUMERIC); $primitiveTransformation = (new PrimitiveTransformation()) ->setCryptoReplaceFfxFpeConfig($cryptoReplaceFfxFpeConfig); // Specify field to be decrypted. $encryptedFields = array_map(function ($encryptedFieldName) { return (new FieldId()) ->setName($encryptedFieldName); }, explode(',', $encryptedFieldNames)); // Associate the decryption with the specified field. $fieldTransformation = (new FieldTransformation()) ->setPrimitiveTransformation($primitiveTransformation) ->setFields($encryptedFields); $recordtransformations = (new RecordTransformations()) ->setFieldTransformations([$fieldTransformation]); $reidentifyConfig = (new DeidentifyConfig()) ->setRecordTransformations($recordtransformations); // Run request. $reidentifyContentRequest = (new ReidentifyContentRequest()) ->setParent($parent) ->setReidentifyConfig($reidentifyConfig) ->setItem($content); $response = $dlp->reidentifyContent($reidentifyContentRequest); // Print the results. $csvRef = fopen($outputCsvFile, 'w'); fputcsv($csvRef, $csvHeaders); foreach ($response->getItem()->getTable()->getRows() as $tableRow) { $values = array_map(function ($tableValue) { return $tableValue->getStringValue(); }, iterator_to_array($tableRow->getValues())); fputcsv($csvRef, $values); }; printf('Table after re-identification (File Location): %s', $outputCsvFile);}Python
To learn how to install and use the client library for Sensitive Data Protection, seeSensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.
importbase64fromtypingimportListimportgoogle.cloud.dlpdefreidentify_table_with_fpe(project:str,table_header:List[str],table_rows:List[List[str]],reid_field_names:List[str],key_name:str=None,wrapped_key:bytes=None,alphabet:str=None,)->None:"""Uses the Data Loss Prevention API to re-identify sensitive data in a table that was encrypted by Format Preserving Encryption (FPE). Args: project: The Google Cloud project id to use as a parent resource. table_header: List of strings representing table field names. table_rows: List of rows representing table data. reid_field_names: A list of fields in table to re-identify. key_name: The name of the Cloud KMS key used to encrypt ('wrap') the AES-256 key. Example: key_name = 'projects/YOUR_GCLOUD_PROJECT/locations/YOUR_LOCATION/ keyRings/YOUR_KEYRING_NAME/cryptoKeys/YOUR_KEY_NAME' wrapped_key: The decrypted ('wrapped', in bytes) AES-256 key to use. This key should be encrypted using the Cloud KMS key specified by key_name. alphabet: The set of characters to replace sensitive ones with. For more information, see https://cloud.google.com/dlp/docs/reference/ rest/v2/projects.deidentifyTemplates#ffxcommonnativealphabet """# Instantiate a client.dlp=google.cloud.dlp_v2.DlpServiceClient()# Construct the `table`. For more details on the table schema, please see# https://cloud.google.com/dlp/docs/reference/rest/v2/ContentItem#Tableheaders=[{"name":val}forvalintable_header]rows=[]forrowintable_rows:rows.append({"values":[{"string_value":cell_val}forcell_valinrow]})table={"headers":headers,"rows":rows}# Convert table to `item`item={"table":table}# Specify fields to be re-identified/decrypted.reid_field_names=[{"name":_i}for_iinreid_field_names]# Construct FPE configuration dictionarycrypto_replace_ffx_fpe_config={"crypto_key":{"kms_wrapped":{"wrapped_key":wrapped_key,"crypto_key_name":key_name}},"common_alphabet":alphabet,}# Construct reidentify configuration dictionaryreidentify_config={"record_transformations":{"field_transformations":[{"primitive_transformation":{"crypto_replace_ffx_fpe_config":crypto_replace_ffx_fpe_config,},"fields":reid_field_names,}]}}# Convert the project id into a full resource id.parent=f"projects/{project}/locations/global"# Call the API.response=dlp.reidentify_content(request={"parent":parent,"reidentify_config":reidentify_config,"item":item,})# Print out results.print(f"Table after re-identification:{response.item.table}")Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-12-17 UTC.