Repository files navigation

Command line tool to rearrange CSV files in certain ways.

This was developed to help transformingSimplified Chinese survey result of2020 State of Rust survey to match the global version.

Firstly, with a CSV file, use the following command to extract all the columns:

csv-transformer extract original.csv> transform.yaml

This will transform a CSV file like

Question 1,Question 2,Question 3Answer B,Answer X,Answer A,Answer X,Random textAnswer A,Answer Y,

into

-"A: Question 1"-"B: Question 2"-"C: Question 3"

The strings formattedX: Header text is called a column reference,and the letters before the first colon is the column index.

You can then edit the YAML file to reflect the transformation you want.Please refer to thetransformations section for available transformations.This is an example of transformation file we used for the survey:transform.yaml.

After you edit it, you can use the following command to generate the result:

csv-transformer transform original.csv transform.yaml> result.csv

Each item in the YAML file represents a ruleto generate one or more columns in the transformation result in its order.If it's kept untouched (just a column reference),the column would be preserved as is.Otherwise, it can be one of the following transformations.

A rename transformation basically just changes the header text.

Example:

-transform:renameheader:"New Header"column:"A: Old Header"

transforms

to

A timestamp transformation reformats the date and time valuewith the format ofstrftime.

Example:

-transform:timestampcolumn:"A: Timestamp"from:"%d-%b-%Y %H:%M:%S"to:"%d/%m/%Y %H:%M:%S"

transforms

to

Optionally, you can also provide aheader field to rename the column at the same time.

A join transformation concatenates values from multiple columns into a single column.

Example:

-transform:joinheader:"Question?"columns:  -"A: Question? Rust"  -"B: Question? C++"  -"C: Question? Python"

transforms

to

Optionally, you can provide asep field to change the default separator, to something else.

It's also possible to slightly format the values from columns before joiningvia replacing the column reference item with a object.

Example:

-transform:joinheader:"Conference anywhere?"columns:  -column:"A: Conference in China?"format:"China - {}"  -column:"B: Conference outside China?"format:"outside China - {}"

transforms

to

A transpose transformation transposes values and their header across several columns.

Example:

-transform:transposesources:"A: Question? 1st":1st"B: Question? 2nd":2nd"C: Question? 3rd":3rdcolumns:"Question? Go":Go"Question? C++":C++"Question? Rust":Rust"Question? Python":Python

transforms

to

If a value is present in multiple source columns,the first matching one would be picked.

An error would be raised if a non-empty value in the source columns can't be mapped to a target column.

All transformations are insrc/transform directory, and new transformations can be added there.

Copyright (C) 2020 Xidorn Quan

This program is free software: you can redistribute it and/or modifyit under the terms of the GNU General Public License as published bythe Free Software Foundation, either version 3 of the License, or(at your option) any later version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY WARRANTY; without even the implied warranty ofMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See theGNU General Public License for more details.

You should have received a copy of the GNU General Public Licensealong with this program. If not, seehttps://www.gnu.org/licenses/.