Repository files navigation

Use the new Node.js Buffer APIs (Buffer.from,Buffer.alloc,Buffer.allocUnsafe,Buffer.allocUnsafeSlow) in all versions of Node.js.

Uses the built-in implementation when available.

npm install safe-buffer

The goal of this package is to provide a safe replacement for the node.jsBuffer.

It's a drop-in replacement forBuffer. You can use it by adding onerequire line tothe top of your node.js modules:

varBuffer=require('safe-buffer').Buffer// Existing buffer code will continue to work without issues:newBuffer('hey','utf8')newBuffer([1,2,3],'utf8')newBuffer(obj)newBuffer(16)// create an uninitialized buffer (potentially unsafe)// But you can use these new explicit APIs to make clear what you want:Buffer.from('hey','utf8')// convert from many types to a BufferBuffer.alloc(16)// create a zero-filled buffer (safe)Buffer.allocUnsafe(16)// create an uninitialized buffer (potentially unsafe)

• array {Array}

Allocates a newBuffer using anarray of octets.

constbuf=Buffer.from([0x62,0x75,0x66,0x66,0x65,0x72]);// creates a new Buffer containing ASCII bytes// ['b','u','f','f','e','r']

ATypeError will be thrown ifarray is not anArray.

• arrayBuffer {ArrayBuffer} The.buffer property of aTypedArray oranew ArrayBuffer()
• byteOffset {Number} Default:0
• length {Number} Default:arrayBuffer.length - byteOffset

When passed a reference to the.buffer property of aTypedArray instance,the newly createdBuffer will share the same allocated memory as theTypedArray.

constarr=newUint16Array(2);arr[0]=5000;arr[1]=4000;constbuf=Buffer.from(arr.buffer);// shares the memory with arr;console.log(buf);// Prints: <Buffer 88 13 a0 0f>// changing the TypedArray changes the Buffer alsoarr[1]=6000;console.log(buf);// Prints: <Buffer 88 13 70 17>

The optionalbyteOffset andlength arguments specify a memory range withinthearrayBuffer that will be shared by theBuffer.

constab=newArrayBuffer(10);constbuf=Buffer.from(ab,0,2);console.log(buf.length);// Prints: 2

ATypeError will be thrown ifarrayBuffer is not anArrayBuffer.

• buffer {Buffer}

Copies the passedbuffer data onto a newBuffer instance.

constbuf1=Buffer.from('buffer');constbuf2=Buffer.from(buf1);buf1[0]=0x61;console.log(buf1.toString());// 'auffer'console.log(buf2.toString());// 'buffer' (copy is not changed)

ATypeError will be thrown ifbuffer is not aBuffer.

• str {String} String to encode.
• encoding {String} Encoding to use, Default:'utf8'

Creates a newBuffer containing the given JavaScript stringstr. Ifprovided, theencoding parameter identifies the character encoding.If not provided,encoding defaults to'utf8'.

constbuf1=Buffer.from('this is a tést');console.log(buf1.toString());// prints: this is a téstconsole.log(buf1.toString('ascii'));// prints: this is a tC)stconstbuf2=Buffer.from('7468697320697320612074c3a97374','hex');console.log(buf2.toString());// prints: this is a tést

ATypeError will be thrown ifstr is not a string.

• size {Number}
• fill {Value} Default:undefined
• encoding {String} Default:utf8

Allocates a newBuffer ofsize bytes. Iffill isundefined, theBuffer will bezero-filled.

constbuf=Buffer.alloc(5);console.log(buf);// <Buffer 00 00 00 00 00>

Thesize must be less than or equal to the value ofrequire('buffer').kMaxLength (on 64-bit architectures,kMaxLength is(2^31)-1). Otherwise, a [RangeError][] is thrown. A zero-length Buffer willbe created if asize less than or equal to 0 is specified.

Iffill is specified, the allocatedBuffer will be initialized by callingbuf.fill(fill). See [buf.fill()][] for more information.

constbuf=Buffer.alloc(5,'a');console.log(buf);// <Buffer 61 61 61 61 61>

If bothfill andencoding are specified, the allocatedBuffer will beinitialized by callingbuf.fill(fill, encoding). For example:

constbuf=Buffer.alloc(11,'aGVsbG8gd29ybGQ=','base64');console.log(buf);// <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64>

CallingBuffer.alloc(size) can be significantly slower than the alternativeBuffer.allocUnsafe(size) but ensures that the newly createdBuffer instancecontents willnever contain sensitive data.

ATypeError will be thrown ifsize is not a number.

• size {Number}

Allocates a newnon-zero-filledBuffer ofsize bytes. Thesize mustbe less than or equal to the value ofrequire('buffer').kMaxLength (on 64-bitarchitectures,kMaxLength is(2^31)-1). Otherwise, a [RangeError][] isthrown. A zero-length Buffer will be created if asize less than or equal to0 is specified.

The underlying memory forBuffer instances created in this way isnotinitialized. The contents of the newly createdBuffer are unknown andmay contain sensitive data. Use [buf.fill(0)][] to initialize suchBuffer instances to zeroes.

constbuf=Buffer.allocUnsafe(5);console.log(buf);// <Buffer 78 e0 82 02 01>// (octets will be different, every time)buf.fill(0);console.log(buf);// <Buffer 00 00 00 00 00>

ATypeError will be thrown ifsize is not a number.

Note that theBuffer module pre-allocates an internalBuffer instance ofsizeBuffer.poolSize that is used as a pool for the fast allocation of newBuffer instances created usingBuffer.allocUnsafe(size) (and the deprecatednew Buffer(size) constructor) only whensize is less than or equal toBuffer.poolSize >> 1 (floor ofBuffer.poolSize divided by two). The defaultvalue ofBuffer.poolSize is8192 but can be modified.

Use of this pre-allocated internal memory pool is a key difference betweencallingBuffer.alloc(size, fill) vs.Buffer.allocUnsafe(size).fill(fill).Specifically,Buffer.alloc(size, fill) willnever use the internal Bufferpool, whileBuffer.allocUnsafe(size).fill(fill)will use the internalBuffer pool ifsize is less than or equal to halfBuffer.poolSize. Thedifference is subtle but can be important when an application requires theadditional performance thatBuffer.allocUnsafe(size) provides.

• size {Number}

Allocates a newnon-zero-filled and non-pooledBuffer ofsize bytes. Thesize must be less than or equal to the value ofrequire('buffer').kMaxLength (on 64-bit architectures,kMaxLength is(2^31)-1). Otherwise, a [RangeError][] is thrown. A zero-length Buffer willbe created if asize less than or equal to 0 is specified.

The underlying memory forBuffer instances created in this way isnotinitialized. The contents of the newly createdBuffer are unknown andmay contain sensitive data. Use [buf.fill(0)][] to initialize suchBuffer instances to zeroes.

When usingBuffer.allocUnsafe() to allocate newBuffer instances,allocations under 4KB are, by default, sliced from a single pre-allocatedBuffer. This allows applications to avoid the garbage collection overhead ofcreating many individually allocated Buffers. This approach improves bothperformance and memory usage by eliminating the need to track and cleanup asmanyPersistent objects.

However, in the case where a developer may need to retain a small chunk ofmemory from a pool for an indeterminate amount of time, it may be appropriateto create an un-pooled Buffer instance usingBuffer.allocUnsafeSlow() thencopy out the relevant bits.

// need to keep around a few small chunks of memoryconststore=[];socket.on('readable',()=>{constdata=socket.read();// allocate for retained dataconstsb=Buffer.allocUnsafeSlow(10);// copy the data into the new allocationdata.copy(sb,0,0,10);store.push(sb);});

Use ofBuffer.allocUnsafeSlow() should be used only as a last resortaftera developer has observed undue memory retention in their applications.

ATypeError will be thrown ifsize is not a number.

The rest of theBuffer API is exactly the same as in node.js.See the docs.

• Node.js issue: Buffer(number) is unsafe
• Node.js Enhancement Proposal: Buffer.from/Buffer.alloc/Buffer.zalloc/Buffer() soft-deprecate

Today, the node.jsBuffer constructor is overloaded to handle many different argumenttypes likeString,Array,Object,TypedArrayView (Uint8Array, etc.),ArrayBuffer, and alsoNumber.

The API is optimized for convenience: you can throw any type at it, and it will try to dowhat you want.

Because the Buffer constructor is so powerful, you often see code like this:

// Convert UTF-8 strings to hexfunctiontoHex(str){returnnewBuffer(str).toString('hex')}

But what happens iftoHex is called with aNumber argument?

If an attacker can make your program call theBuffer constructor with aNumberargument, then they can make it allocate uninitialized memory from the node.js process.This could potentially disclose TLS private keys, user data, or database passwords.

When theBuffer constructor is passed aNumber argument, it returns anUNINITIALIZED block of memory of the specifiedsize. When you create aBuffer likethis, youMUST overwrite the contents before returning it to the user.

From thenode.js docs:

new Buffer(size)

• size Number

The underlying memory forBuffer instances created in this way is not initialized.The contents of a newly createdBuffer are unknown and could contain sensitivedata. Usebuf.fill(0) to initialize a Buffer to zeroes.

(Emphasis our own.)

Whenever the programmer intended to create an uninitializedBuffer you often see codelike this:

varbuf=newBuffer(16)// Immediately overwrite the uninitialized buffer with data from another bufferfor(vari=0;i<buf.length;i++){buf[i]=otherBuf[i]}

Yes. It's surprisingly common to forget to check the type of your variables in adynamically-typed language like JavaScript.

Usually the consequences of assuming the wrong type is that your program crashes with anuncaught exception. But the failure mode for forgetting to check the type of arguments totheBuffer constructor is more catastrophic.

Here's an example of a vulnerable service that takes a JSON payload and converts it tohex:

// Take a JSON payload {str: "some string"} and convert it to hexvarserver=http.createServer(function(req,res){vardata=''req.setEncoding('utf8')req.on('data',function(chunk){data+=chunk})req.on('end',function(){varbody=JSON.parse(data)res.end(newBuffer(body.str).toString('hex'))})})server.listen(8080)

In this example, an http client just has to send:

{"str":1000}

and it will get back 1,000 bytes of uninitialized memory from the server.

This is a very serious bug. It's similar in severity to thethe Heartbleed bug that allowed disclosure of OpenSSL processmemory by remote attackers.

Mathias Buus and I(Feross Aboukhadijeh) found this issue in one of our own packages,bittorrent-dht. The bug would allowanyone on the internet to send a series of messages to a user ofbittorrent-dht and getthem to reveal 20 bytes at a time of uninitialized memory from the node.js process.

Here'sthe committhat fixed it. We released a new fixed version, created aNode Security Project disclosure, and deprecated allvulnerable versions on npm so users will get a warning to upgrade to a newer version.

That got us wondering if there were other vulnerable packages. Sure enough, within a shortperiod of time, we found the same issue inws, themost popular WebSocket implementation in node.js.

If certain APIs were called withNumber parameters instead ofString orBuffer asexpected, then uninitialized server memory would be disclosed to the remote peer.

These were the vulnerable methods:

socket.send(number)socket.ping(number)socket.pong(number)

Here's a vulnerable socket server with some echo functionality:

server.on('connection',function(socket){socket.on('message',function(message){message=JSON.parse(message)if(message.type==='echo'){socket.send(message.data)// send back the user's message}})})

socket.send(number) called on the server, will disclose server memory.

Here'sthe release where the issuewas fixed, with a more detailed explanation. Props toArnout Kazemier for the quick fix. Here's theNode Security Project disclosure.

It's important that node.js offers a fast way to get memory otherwise performance-criticalapplications would needlessly get a lot slower.

But we need a better way tosignal our intent as programmers.When we wantuninitialized memory, we should request it explicitly.

Sensitive functionality should not be packed into a developer-friendly API that looselyaccepts many different types. This type of API encourages the lazy practice of passingvariables in without checking the type very carefully.

The functionality of creating buffers with uninitialized memory should be part of anotherAPI. We proposeBuffer.allocUnsafe(number). This way, it's not part of an API thatfrequently gets user input of all sorts of different types passed into it.

varbuf=Buffer.allocUnsafe(16)// careful, uninitialized memory!// Immediately overwrite the uninitialized buffer with data from another bufferfor(vari=0;i<buf.length;i++){buf[i]=otherBuf[i]}

We senta PR to node.js core (merged assemver-major) which defends against one case:

varstr=16newBuffer(str,'utf8')

In this situation, it's implied that the programmer intended the first argument to be astring, since they passed an encoding as a second argument. Today, node.js will allocateuninitialized memory in the case ofnew Buffer(number, encoding), which is probably notwhat the programmer intended.

But this is only a partial solution, since if the programmer doesnew Buffer(variable)(without anencoding parameter) there's no way to know what they intended. Ifvariableis sometimes a number, then uninitialized memory will sometimes be returned.

We could deprecate and removenew Buffer(number) and useBuffer.allocUnsafe(number) whenwe need uninitialized memory. But that would break 1000s of packages.

We believe the best solution is to:

1. Changenew Buffer(number) to return safe, zeroed-out memory

2. Create a new API for creating uninitialized Buffers. We propose:Buffer.allocUnsafe(number)

We now support adding three new APIs:

• Buffer.from(value) - convert from any type to a buffer
• Buffer.alloc(size) - create a zero-filled buffer
• Buffer.allocUnsafe(size) - create an uninitialized buffer with given size

This solves the core problem that affectedws andbittorrent-dht which isBuffer(variable) getting tricked into taking a number argument.

This way, existing code continues working and the impact on the npm ecosystem will beminimal. Over time, npm maintainers can migrate performance-critical code to useBuffer.allocUnsafe(number) instead ofnew Buffer(number).

We think there's a serious design issue with theBuffer API as it exists today. Itpromotes insecure software by putting high-risk functionality into a convenient APIwith friendly "developer ergonomics".

This wasn't merely a theoretical exercise because we found the issue in some of themost popular npm packages.

Fortunately, there's an easy fix that can be applied today. Usesafe-buffer in place ofbuffer.

varBuffer=require('safe-buffer').Buffer

Eventually, we hope that node.js core can switch to this new, safer behavior. We believethe impact on the ecosystem would be minimal since it's not a breaking change.Well-maintained, popular packages would be updated to useBuffer.alloc quickly, whileolder, insecure packages would magically become safe from this attack vector.

• Node.js PR: buffer: throw if both length and enc are passed
• Node Security Project disclosure forws
• Node Security Project disclosure forbittorrent-dht

The original issues inbittorrent-dht(disclosure) andws (disclosure) were discovered byMathias Buus andFeross Aboukhadijeh.

Thanks toAdam Baldwin for helping disclose these issuesand for his work running theNode Security Project.

Thanks toJohn Hiesey for proofreading this README andauditing the code.

MIT. Copyright (C)Feross Aboukhadijeh