Repository files navigation

CSVdata is anpm module forNodeJS, thatloads, writes, and checks data operating with CSV files. Based onnode-csv, supports native JS promises and streams (requires Node >= v6.4.0). It has a simple API, it is well tested and built for high performance.

It includes some smart features to try preventing common errors that could compromise data integrity when dealing with CSV (e.g. mixing of values due to a missing entry).

npm install csvdata

(add "--save" if you want the module to be automatically added to your project's "package.json" dependencies)

const csvdata = require('csvdata')

csvdata.load(filePath, [options])

Reads data from "filePath" (the first line of the CSV file must contain headers).

Returns a promise, eventually fulfilled with an array where each item is an object that contains data from a row (automatically parses native JS data types).

The"options" argument is a configuration object with the following default values.

{delimiter:',',encoding:'utf8',log:true,objName:undefined,parse:true,stream:false}

• delimiter (string): set the field delimiter (one character only).

• encoding (string): set the file encoding (must besupported by Node.js).

• log (boolean): if set tofalse disable logs.

• objName (string): instead of an array it returns an "index" object, where keys map to each entry in the column titled "objName", and values are objects that contain all data from the corresponding row (meant to be used when entries in the column "objName" are unique, and faster retrieval is convenient).

• parse (boolean): whether to automatically parse data to native JS types or not (e.g. it would convert the string '07.23' to the number '7.23').

• stream (boolean): if set totrue, it returns areadable stream that can be piped where needed.

// Imagine the CSV file content is:// name,hair,age// John,brown,36// Laura,red,23// Boris,blonde,28//csvdata.load('./my-file.csv')// -> Returns a promise that will be fulfilled with:// [//   {name: 'John', hair: 'brown', age: 36},//   {name: 'Laura', hair: 'red', age: 23},//   {name: 'Boris', hair: 'blonde', age: 28}// ]csvdata.load('./my-file.csv',{objName:'name'})// -> Returns a promise that will be fulfilled with:// {//   John: {name: 'John', hair: 'brown', age: 36},//   Laura: {name: 'Laura', hair: 'red', age: 23},//   Boris: {name: 'Boris', hair: 'blonde', age: 28}// }

csvdata.write(filePath, data, [options])

Returns a promise, eventually fulfilled when done writing data to "filePath" (be careful, as it overwrites existing files). Data can be provided as:

• String (e.g.'a,b,c\nd,e,f')
• Array of arrays (e.g.[['a','b','c'],['d','e','f']])
• Array of objects (e.g.[{amount: 100, name: 'John'}, {amount: 130, name: 'Paul'}])
• Object containing objects (e.g.{John: {amount: '100', name: 'John' }, Paul: {amount: '130', name: 'Paul'}}).

The"options" argument is a configuration object with the following default values.

{append:false,delimiter:',',empty:false,encoding:'utf8',header:'',log:true}

• append (boolean): whether to create a new file or append data to an existing one.

• delimiter (string): set the field delimiter (one character only).

• empty (boolean): if set totrue, return an error when the dataset contains empty values (i.e.undefined,null, or'').

• encoding (string): set thefile encoding.

• header (string): if provided it's written on the first line. If data comes from an object (i.e. last two cases above), "header"must be provided to guarantee the correct order of comma separated values, and can be used toselect which object properties are saved to CSV.

• log (boolean): if set tofalse disable logs.

vardata=[{name:'John',hair:'brown',age:36},{name:'Laura',hair:'red',age:23},{name:'Boris',hair:undefined,age:28}];csvdata.write('./my-file.csv',data,{header:'name,hair,age'})// Generates "my-file.csv" with this content:// name,hair,age// John,brown,36// Laura,red,23// Boris,,28//csvdata.write('./my-file.csv',data,{header:'age,hair,name'})// Generates "my-file.csv" with this content:// age,hair,name// 36,brown,John// 23,red,Laura// 28,,Boris//csvdata.write('./my-file.csv',data,{empty:true,header:'name,hair,age'})// -> Rejects the promise with an error.// Empty value "hair" in object:// {"name":"Boris","age":28}

csvdata.check(filePath, [options])

Checks data integrity of the CSV file. It can look for missing, empty, and duplicate values within columns, or detect empty lines.

Returns a promise, eventually fulfilled withtrue if the check is ok, orfalse if there are any problems (before using this method, make sure the first line – i.e. the header – of your CSV file is correct).

The"options" argument is a configuration object with the following default values.

{delimiter:',',duplicates:false,emptyLines:false,emptyValues:true,encoding:'utf8',limit:false,log:true}

• delimiter (string): set the field delimiter (one character only).

• duplicates (boolean): check for duplicate values within columns.

• emptyLines (boolean) check for empty lines.

• emptyValues (boolean) check for empty values. If set tofalse it considers empty values fine, but still complains for missing values.

• encoding (string): set the file encoding (must besupported by Node.js).

• limit (string): comma separated column headers, if provided limit the "duplicates" and "emptyValues" checks to a subset of columns (instead missing values and empty lines can only be checked for the whole file, due to the CSV format).

• log (boolean): if set tofalse, only the final result is returned. The process becomes faster and requires less memory (as it doesn't need to keep track of where the problems occur).

Note that checking for duplicate values requires to load the selected CSV content in memory, as the program needs to have a reference to previous values (this might be an issue if you're dealing with very large files, that exceed your available memory).

// Imagine the CSV file content is:// name,hair,age// John,brown,36// Laura,red// Boris,,36// Laura,black,//csvdata.check('./my-file.csv')// -> Returns a promise that will be fulfilled with "false".// (also logs)// - Missing value on line 3// - Empty value on line:// 4 (hair)// 5 (age)csvdata.check('./my-file.csv',{emptyValues:false})// -> Returns a promise that will be fulfilled with "false".// (also logs)// - Missing value on line 3csvdata.check('./my-file.csv',{duplicates:true})// -> Returns a promise that will be fulfilled with "false".// (also logs)// - Missing value on line 3// - Duplicate values for "name":// "Laura" on line 3, 5csvdata.check('./my-file.csv',{duplicates:true,limit:'hair,age'})// -> Returns a promise that will be fulfilled with "false".// (also logs)// - Missing value on line 3// - Empty value on line:// 4 (hair)// 5 (age)csvdata.check('./my-file.csv',{log:false})// -> Returns a promise that will be fulfilled with "false".// Not logging is faster if you need just the final result.

The "check" method can also be executed from the command line.

# You can run it either asnode csvdata.js -c<your_file_path.csv># Or make the file executable withchmod +x csvdata.js# And then run it as./csvdata.js -c<your_file_path.csv># To see the other options, check command line helpnode csvdata.js -h

MIT License