- Notifications
You must be signed in to change notification settings - Fork0
Anonymizer tool for datasets such CSV files
License
fgmacedo/datanonymizer
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Anonymizer tool for datasets such CSV files.
To generate fake data, you can choose between two excelent generators:
Using pip:
pip install datanonymizer
Using mimesis instead of the default Faker:
pip install datanonymizer[mimesis]
Or from source:
git clone https://github.com/fgmacedo/datanonymizercd datanonymizerpython setup.py installPass your data throughstdin and get it back anonymized onstdout.
Note
In this case, the output will be equal to the input as no conversions were applied.
cat input_file.csv| datanonymizer>output_file.csv
Using a config file to declare conversions and generators for the required fields:
cat input_file.csv| datanonymizer --config ./dataset_anon_config.yml>output_file.csv
Please see examples folder for a small demo:
cat examples/small.csv| python -m datanonymizer -i --config examples/small_faker.yml --seed my_seed>examples/small_anonymized_using_faker.csv
Optional arguments:
-h, --help show this help message and exit-l LANGUAGE, --language LANGUAGE Language used by the Generator-di DELIMITER_INPUT, --delimiter_input DELIMITER_INPUT CSV delimiter-do DELIMITER_OUTPUT, --delimiter_output DELIMITER_OUTPUT CSV delimiter-i, --ignore_errors Continue on errors--head HEAD Outputs only the first <HEAD> lines-g {faker,mimesis}, --generator {faker,mimesis} Generator library to be used for fake data--seed SEED Seed for the pseudo random generator providers--config CONFIG Configuration fileYou'l need a configuration file to setup transformations for each dataset.
This file is a simpleyaml where you can configure fields.
Field names should match the column name declared into the CSV input file.
---fields:Task ID:omit:trueLocation:conversions: -fn:coords_to_h3kwargs:resolution:8Client Address:conversions: -fn:has_valuerename:has_addressCompany Name:generator:provider:business.companyrename:companyInvoice ID:generator:provider:person.identifierkwargs:mask:"#######"rename:invoice
The generatos clause depends of the library you choose to provide fake data.
You can use any generator available at the generic API from Faker or mimesis.
For example, if you wanna mimic data with company names:
Faker
---fields:Company Name:generator:provider:company
Mimesis
---fields:Company Name:generator:provider:business.company
But you can replace the real names by names of fruits (using Mimesis) or any other provider:
---fields:Company Name:generator:provider:food.fruit
Or generate random integers to replace real IDs:
Faker
---fields:ID:generator:provider:pyintkwargs:min_value:1max_value:15_000_000
Mimesis
---fields:ID:generator:provider:person.identifierkwargs:mask:"#######"
You can apply any pre-configured conversion functions available.
- coords_to_h3
- has_value
About
Anonymizer tool for datasets such CSV files
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.