Repository files navigation

A Node.js module for parsing form data, especially file uploads.

If you have anyhow-to kind of questions, please read theContributingGuide andCode of Conductdocuments.
For bugs reports and feature requests,please create anissue or ping@wgw_eth / @wgw_lolat Twitter.

This project issemantically versioned and if you want support in migrating between versions you can schedule us for training or support us through donations, so we can prioritize.

This module was initially developed by@felixge forTransloadit, a service focused on uploading andencoding images and videos. It has been battle-tested against hundreds of GBs offile uploads from a large variety of clients and is considered production-readyand is used in production for years.

Currently, we are few maintainers trying to deal with it. :) More contributorsare always welcome! ❤️ Jump onissue #412 which isclosed, but if you are interested we can discuss it and add you after strictrules, like enabling Two-Factor Auth in your npm and GitHub accounts.

• Fast (~900-2500 mb/sec) & streaming multipart parser
• Automatically writing file uploads to disk (optional, seeoptions.fileWriteStreamHandler)
• Plugins API - allowing custom parsers and plugins
• Low memory footprint
• Graceful error handling
• Very high test coverage

This package is a dual ESM/commonjs package.

This is a low-level package, and if you're using a high-level framework itmayalready be included. Check the examples below and theexamples/ folder.

# v2npm install formidable@v2# v3npm install formidablenpm install formidable@v3

Note: Future not ready releases will be published on*-next dist-tags for the corresponding version.

For more examples look at theexamples/ directory.

Parse an incoming file upload, with theNode.js's built-inhttp module.

importhttpfrom'node:http';importformidable,{errorsasformidableErrors}from'formidable';constserver=http.createServer(async(req,res)=>{if(req.url==='/api/upload'&&req.method.toLowerCase()==='post'){// parse a file uploadconstform=formidable({});letfields;letfiles;try{[fields,files]=awaitform.parse(req);}catch(err){// example to check for a very specific errorif(err.code===formidableErrors.maxFieldsExceeded){}console.error(err);res.writeHead(err.httpCode||400,{'Content-Type':'text/plain'});res.end(String(err));return;}res.writeHead(200,{'Content-Type':'application/json'});res.end(JSON.stringify({ fields, files},null,2));return;}// show a file upload formres.writeHead(200,{'Content-Type':'text/html'});res.end(`    <h2>With Node.js <code>"http"</code> module</h2>    <form action="/api/upload" enctype="multipart/form-data" method="post">      <div>Text field title: <input type="text" name="title" /></div>      <div>File: <input type="file" name="multipleFiles" multiple="multiple" /></div>      <input type="submit" value="Upload" />    </form>  `);});server.listen(8080,()=>{console.log('Server listening on http://localhost:8080/ ...');});

There are multiple variants to do this, but Formidable just need Node.js Requeststream, so something like the following example should work just fine, withoutany third-partyExpress.js middleware.

Or try theexamples/with-express.js

importexpressfrom'express';importformidablefrom'formidable';constapp=express();app.get('/',(req,res)=>{res.send(`    <h2>With <code>"express"</code> npm package</h2>    <form action="/api/upload" enctype="multipart/form-data" method="post">      <div>Text field title: <input type="text" name="title" /></div>      <div>File: <input type="file" name="someExpressFiles" multiple="multiple" /></div>      <input type="submit" value="Upload" />    </form>  `);});app.post('/api/upload',(req,res,next)=>{constform=formidable({});form.parse(req,(err,fields,files)=>{if(err){next(err);return;}res.json({ fields, files});});});app.listen(3000,()=>{console.log('Server listening on http://localhost:3000 ...');});

Of course, withKoa v1, v2 or future v3 the thingsare very similar. You can useformidable manually as shown below or throughthekoa-better-body package which isusingformidable under the hood and support more features and differentrequest bodies, check its documentation for more info.

Note: this example is assuming Koa v2. Be aware that you should passctx.reqwhich is Node.js's Request, andNOT thectx.request which is Koa's Requestobject - there is a difference.

importKoafrom'Koa';importformidablefrom'formidable';constapp=newKoa();app.on('error',(err)=>{console.error('server error',err);});app.use(async(ctx,next)=>{if(ctx.url==='/api/upload'&&ctx.method.toLowerCase()==='post'){constform=formidable({});// not very elegant, but that's for now if you don't want to use `koa-better-body`// or other middlewares.awaitnewPromise((resolve,reject)=>{form.parse(ctx.req,(err,fields,files)=>{if(err){reject(err);return;}ctx.set('Content-Type','application/json');ctx.status=200;ctx.state={ fields, files};ctx.body=JSON.stringify(ctx.state,null,2);resolve();});});awaitnext();return;}// show a file upload formctx.set('Content-Type','text/html');ctx.status=200;ctx.body=`    <h2>With <code>"koa"</code> npm package</h2>    <form action="/api/upload" enctype="multipart/form-data" method="post">    <div>Text field title: <input type="text" name="title" /></div>    <div>File: <input type="file" name="koaFiles" multiple="multiple" /></div>    <input type="submit" value="Upload" />    </form>  `;});app.use((ctx)=>{console.log('The next middleware is called');console.log('Results:',ctx.state);});app.listen(3000,()=>{console.log('Server listening on http://localhost:3000 ...');});

The benchmark is quite old, from the old codebase. But maybe quite true though.Previously the numbers was around ~500 mb/sec. Currently with moving to the newNode.js Streams API it's faster. You can clearly see the differences between theNode versions.

Note: a lot better benchmarking could and should be done in future.

Benchmarked on 8GB RAM, Xeon X3440 (2.53 GHz, 4 cores, 8 threads)

~/github/node-formidable master❯ nve --parallel 8 10 12 13 node benchmark/bench-multipart-parser.js ⬢  Node 81261.08 mb/sec ⬢  Node 101113.04 mb/sec ⬢  Node 122107.00 mb/sec ⬢  Node 132566.42 mb/sec

All shown are equivalent.

Please passoptions to the function/constructor, not by assigningthem to the instanceform

importformidablefrom'formidable';constform=formidable(options);

See it's defaults insrc/Formidable.js DEFAULT_OPTIONS(theDEFAULT_OPTIONS constant).

• options.encoding{string} - default'utf-8'; sets encoding forincoming form fields,

• options.uploadDir{string} - defaultos.tmpdir(); the directory forplacing file uploads in. You can move them later by usingfs.rename().

• options.keepExtensions{boolean} - defaultfalse; to include theextensions of the original files or not

• options.allowEmptyFiles{boolean} - defaultfalse; allow upload emptyfiles

• options.minFileSize{number} - default1 (1byte); the minium size ofuploaded file.

• options.maxFiles{number} - defaultInfinity;limit the amount of uploaded files, set Infinity for unlimited

• options.maxFileSize{number} - default200 * 1024 * 1024 (200mb);limit the size of each uploaded file.

• options.maxTotalFileSize{number} - defaultoptions.maxFileSize;limit the size of the batch of uploaded files.

• options.maxFields{number} - default1000; limit the number of fields, set Infinity for unlimited

• options.maxFieldsSize{number} - default20 * 1024 * 1024 (20mb);limit the amount of memory all fields together (except files) can allocate inbytes.

• options.hashAlgorithm{string | false} - defaultfalse; include checksums calculatedfor incoming files, set this to some hash algorithm, seecrypto.createHashfor available algorithms

• options.fileWriteStreamHandler{function} - defaultnull, which bydefault writes to host machine file system every file parsed; The functionshould return an instance of aWritable streamthat will receive the uploaded file data. With this option, you can have anycustom behavior regarding where the uploaded file data will be streamed for.If you are looking to write the file uploaded in other types of cloud storages(AWS S3, Azure blob storage, Google cloud storage) or private file storage,this is the option you're looking for. When this option is defined the defaultbehavior of writing the file in the host machine file system is lost.

• options.filename{function} - defaultundefined Use it to controlnewFilename. Must return a string. Will be joined with options.uploadDir.

• options.filter{function} - default function that always returns true.Use it to filter files before they are uploaded. Must return a boolean. Will not make the form.parse error

• options.createDirsFromUploads{boolean} - default false. If true, makes direct folder uploads possible. Use<input type="file" name="folders" webkitdirectory directory multiple> to create a form to upload folders. Has to be used with the optionsoptions.uploadDir andoptions.filename whereoptions.filename has to return a string with the character/ for folders to be created. The base will beoptions.uploadDir.

where part can be decomposed as

const{ originalFilename, mimetype}=part;

Note: If this size of combined fields, or size of some file is exceeded, an'error' event is fired.

// The amount of bytes received for this form so far.form.bytesReceived;

// The expected number of bytes in this form.form.bytesExpected;

Behaves like Array.filter: Returning false will simply ignore the file and go to the next.

constoptions={filter:function({name, originalFilename, mimetype}){// keep only imagesreturnmimetype&&mimetype.includes("image");}};

Note: use an outside variable to cancel all uploads upon the first error

Note: use form.emit('error') to make form.parse error

letcancelUploads=false;// create variable at the same scope as formconstoptions={filter:function({name, originalFilename, mimetype}){// keep only imagesconstvalid=mimetype&&mimetype.includes("image");if(!valid){form.emit('error',newformidableErrors.default('invalid type',0,400));// optional make form.parse errorcancelUploads=true;//variable to make filter return false after the first problem}returnvalid&&!cancelUploads;}};

Parses an incoming Node.jsrequest containing form data. Ifcallback is not provided a promise is returned.

constform=formidable({uploadDir:__dirname});form.parse(req,(err,fields,files)=>{console.log('fields:',fields);console.log('files:',files);});// with Promiseconst[fields,files]=awaitform.parse(req);

You may overwrite this method if you are interested in directly accessing themultipart stream. Doing so will disable any'field' /'file' eventsprocessing which would occur otherwise, making you fully responsible forhandling the processing.

AboutuploadDir, given the following directory structure

project-name├── src│   └── server.js│└── uploads    └── image.jpg

__dirname would be the same directory as the source file itself (src)

`${__dirname}/../uploads`

to put files in uploads.

Omitting__dirname would make the path relative to the current working directory. This would be the same if server.js is launched from src but not project-name.

null will use default which isos.tmpdir()

Note: If the directory does not exist, the uploaded files aresilently discarded. To make sure it exists:

import{createNecessaryDirectoriesSync}from"filesac";constuploadPath=`${__dirname}/../uploads`;createNecessaryDirectoriesSync(`${uploadPath}/x`);

In the example below, we listen on couple of events and direct them to thedata listener, so you can do whatever you choose there, based on whether itsbefore the file been emitted, the header value, the header name, on field, onfile and etc.

Or the other way could be to just override theform.onPart as it's shown a bitlater.

form.once('error',console.error);form.on('fileBegin',(formname,file)=>{form.emit('data',{name:'fileBegin', formname,value:file});});form.on('file',(formname,file)=>{form.emit('data',{name:'file', formname,value:file});});form.on('field',(fieldName,fieldValue)=>{form.emit('data',{name:'field',key:fieldName,value:fieldValue});});form.once('end',()=>{console.log('Done!');});// If you want to customize whatever you want...form.on('data',({ name, key, value, buffer, start, end, formname, ...more})=>{if(name==='partBegin'){}if(name==='partData'){}if(name==='headerField'){}if(name==='headerValue'){}if(name==='headerEnd'){}if(name==='headersEnd'){}if(name==='field'){console.log('field name:',key);console.log('field value:',value);}if(name==='file'){console.log('file:',formname,value);}if(name==='fileBegin'){console.log('fileBegin:',formname,value);}});

A method that allows you to extend the Formidable library. By default we include4 plugins, which essentially are adapters to plug the different built-in parsers.

The plugins added by this method are always enabled.

Seesrc/plugins/ for more detailed look on default plugins.

Theplugin param has such signature:

function(formidable:Formidable,options:Options):void;

The architecture is simple. Theplugin is a function that is passed with theFormidable instance (theform across the README examples) and the options.

Note: the plugin function'sthis context is also the same instance.

constform=formidable({keepExtensions:true});form.use((self,options)=>{// self === this === formconsole.log('woohoo, custom plugin');// do your stuff; check `src/plugins` for inspiration});form.parse(req,(error,fields,files)=>{console.log('done!');});

Important to note, is that inside pluginthis.options,self.options andoptions MAY or MAY NOT be the same. General best practice is to always use thethis, so you can later test your plugin independently and more easily.

If you want to disable some parsing capabilities of Formidable, you can disablethe plugin which corresponds to the parser. For example, if you want to disablemultipart parsing (so thesrc/parsers/Multipart.jswhich is used insrc/plugins/multipart.js), thenyou can remove it from theoptions.enabledPlugins, like so

importformidable,{octetstream,querystring,json}from"formidable";constform=formidable({hashAlgorithm:'sha1',enabledPlugins:[octetstream,querystring,json],});

Be aware that the orderMAY be important too. The names corresponds 1:1 tofiles insrc/plugins/ folder.

Pull requests for new built-in plugins MAY be accepted - for example, moreadvanced querystring parser. Add your plugin as a new file insrc/plugins/folder (lowercased) and follow how the other plugins are made.

If you want to use Formidable to only handle certain parts for you, you can dosomething similar. Or see#387 forinspiration, you can for example validate the mime-type.

constform=formidable();form.onPart=(part)=>{part.on('data',(buffer)=>{// do whatever you want here});};

For example, force Formidable to be used only on non-file "parts" (i.e., htmlfields)

constform=formidable();form.onPart=function(part){// let formidable handle only non-file partsif(part.originalFilename===''||!part.mimetype){// used internally, please do not override!form._handlePart(part);}};

exportinterfaceFile{// The size of the uploaded file in bytes.// If the file is still being uploaded (see `'fileBegin'` event),// this property says how many bytes of the file have been written to disk yet.file.size:number;// The path this file is being written to. You can modify this in the `'fileBegin'` event in// case you are unhappy with the way formidable generates a temporary path for your files.file.filepath:string;// The name this file had according to the uploading client.file.originalFilename:string|null;// calculated based on options providedfile.newFilename:string|null;// The mime type of this file, according to the uploading client.file.mimetype:string|null;// A Date object (or `null`) containing the time this file was last written to.// Mostly here for compatibility with the [W3C File API Draft](http://dev.w3.org/2006/webapi/FileAPI/).file.mtime:Date|null;file.hashAlgorithm:false||'sha1'|'md5'|'sha256'// If `options.hashAlgorithm` calculation was set, you can read the hex digest out of this var (at the end it will be a string)file.hash:string|object|null;}

This method returns a JSON-representation of the file, allowing you toJSON.stringify() the file which is useful for logging and responding torequests.

Emitted after each incoming chunk of data that has been parsed. Can be used toroll your own progress bar.Warning Use this only for server side progress bar. On the client side better useXMLHttpRequest withxhr.upload.onprogress =

form.on('progress',(bytesReceived,bytesExpected)=>{});

Emitted whenever a field / value pair has been received.

form.on('field',(name,value)=>{});

Emitted whenever a new file is detected in the upload stream. Use this event ifyou want to stream the file to somewhere else while buffering the upload on thefile system.

form.on('fileBegin',(formName,file)=>{// accessible here// formName the name in the form (<input name="thisname" type="file">) or http filename for octetstream// file.originalFilename http filename or null if there was a parsing error// file.newFilename generated hexoid or what options.filename returned// file.filepath default pathname as per options.uploadDir and options.filename// file.filepath = CUSTOM_PATH // to change the final path});

Emitted whenever a field / file pair has been received.file is an instance ofFile.

form.on('file',(formname,file)=>{// same as fileBegin, except// it is too late to change file.filepath// file.hash is available if options.hash was used});

Emitted when there is an error processing the incoming form. A request thatexperiences an error is automatically paused, you will have to manually callrequest.resume() if you want the request to continue firing'data' events.

May haveerror.httpCode anderror.code attached.

form.on('error',(err)=>{});

Emitted when the request was aborted by the user. Right now this can be due to a'timeout' or 'close' event on the socket. After this event is emitted, anerror event will follow. In the future there will be a separate 'timeout'event (needs a change in the node core).

form.on('aborted',()=>{});

Emitted when the entire request has been received, and all contained files havefinished flushing to disk. This is a great place for you to send your response.

form.on('end',()=>{});

Gets first values of fields, like pre 3.0.0 without multiples pass in a list of optional exceptions where arrays of strings is still wanted (<select multiple> for example)

import{firstValues}from'formidable/src/helpers/firstValues.js';// ...form.parse(request,async(error,fieldsMultiple,files)=>{if(error){//...}constexceptions=['thisshouldbeanarray'];constfieldsSingle=firstValues(form,fieldsMultiple,exceptions);// ...

Html form input type="checkbox" only send the value "on" if checked,convert it to booleans for each input that is expected to be sent as a checkbox, only use after firstValues or similar was called.

import{firstValues}from'formidable/src/helpers/firstValues.js';import{readBooleans}from'formidable/src/helpers/readBooleans.js';// ...form.parse(request,async(error,fieldsMultiple,files)=>{if(error){//...}constfieldsSingle=firstValues(form,fieldsMultiple);constexpectedBooleans=['checkbox1','wantsNewsLetter','hasACar'];constfieldsWithBooleans=readBooleans(fieldsSingle,expectedBooleans);// ...

./CHANGELOG.md

• multipart-parser: a C++parser based on formidable
• Ryan Dahl for his work onhttp-parser which heavily inspired theinitialmultipart_parser.js.

If the documentation is unclear or has a typo, please click on the page'sEditbutton (pencil icon) and suggest a correction. If you would like to help us fixa bug or add a new feature, please check ourContributingGuide. Pull requests are welcome!

Thanks goes to these wonderful people(emoji key):

Felix Geisendörfer 💻🎨🤔📖	Charlike Mike Reagent 🐛🚇🎨💻📖💡🤔🚧⚠️	Kedar 💻⚠️💬🐛	Walle Cyril 💬🐛💻💵🤔🚧	Xargs 💬🐛💻🚧	Amit-A 💬🐛💻
Charmander 💬🐛💻🤔🚧	Dylan Piercey 🤔	Adam Dobrawy 🐛📖	amitrohatgi 🤔	Jesse Feng 🐛	Nathanael Demacon 💬💻👀
MunMunMiao 🐛	Gabriel Petrovay 🐛💻	Philip Woods 💻🤔	Dmitry Ivonin 📖	Claudio Poli 💻

From aFelix blog post:

• Sven Lito for fixing bugs and merging patches
• egirshov for contributing many improvements to the node-formidable multipart parser
• Andrew Kelley for also helping with fixing bugs and making improvements
• Mike Frey for contributing JSON support

Formidable is licensed under theMIT License.