language:node_js

os:linux

dist:trusty

node_js:

-"stable"

-"11.0"

-"10.0"

matrix:

jobs:

fast_finish:true

allow_failures:

-node_js:'10.0'

sudo:false

cache:

directories:

-node_modules

timeout:1440

before_script:

-command -v npx || npm i -g npx

Convert different types of JavaScript String to/from Uint8Array.

- Convert different types of JavaScript`String` to/from`Uint8Array`.

- Check for[`String` encoding](#string-types-table).

The main target of this library is the Browser, where there is no`Buffer` type.

Node.js is welcome too, except for`toString('base64')` which depends on`btoa`.

See[Node.js equivalents](#nodejs-equivalents).

let processedSting=buffer2str(buffer,false);

let buffer=str2buffer(ut8EncodedStr,true);

let buffer=str2buffer(mbString,true);

letprocessedUtf8Sting=buffer2str(buffer,true);

letprocessedMbString=buffer2str(buffer,true);

let buffer=str2buffer(anyStr);

let processedSting=buffer2str(buffer);

Simple`sha1` function forbrowser that works with`String`, using crypto, compatible with the PHP counterpart:

Simple`sha1` functionusing`crypto`forBrowser, that works with`String` and is compatible with the PHP counterpart:

import {str2buffer,toString }from'string-encode';

constcrypto=scope.crypto||scope.msCrypto;

constsubtle=crypto.subtle||crypto.webkitSubtle;

asyncfunctionsha1(str,enc='hex') {

let buf=str2buffer(str,true);

buf=awaitcrypto.subtle.digest('SHA-1', buf);

buf=awaitsubtle.digest('SHA-1', buf);

buf=newUint8Array(buf);

returntoString.call(buf, enc);

}

How to use this`sha1` function:

awaitsha1('something');// "1af17e73721dbe0c40011b82ed4bb1a7dbe3ce29"

awaitsha1('something');// "1af17e73721dbe0c40011b82ed4bb1a7dbe3ce29"

awaitsha1('something',false);// "\u001añ~sr\u001d¾\f@\u0001\u001b\u0082íK±§ÛãÎ)"

awaitsha1('что-то');// "991fe0590dfec23402d71c0e817bc7a7ab217e2b"

awaitsha1('что-то');// "991fe0590dfec23402d71c0e817bc7a7ab217e2b"

awaitsha1('что-то','base64');// "mR/gWQ3+wjQC1xwOgXvHp6shfis="

Base64 encode/decode a multibyte string:

utf8Decode(atob('4pqUINC40LvQuCDwn5iE'));// "⚔ или 😄"

|`string-encode` in Browser|`Buffer` in Node.js|

| :---| :---|

| str2buffer(str, false)| Buffer.from(str, 'binary')|

| str2buffer(str, true)| Buffer.from(str, 'utf8')|

| hex2buffer(str)| Buffer.from(str, 'hex')|

| str2buffer(atob(str), false)| Buffer.from(str, 'base64')|

| -| -|

| buffer2str(str, false)| Buffer.toString('binary')|

| buffer2str(str, true)| Buffer.toString('utf8')|

| buffer2hex(str)| Buffer.toString('hex')|

| btoa(buffer2str(str, false))| Buffer.toString('base64')|

If you want your`Uint8Array` to be one step closer to the Node.js's`Buffer`,

buf.toString= toString;// the magic method

console.log(buf+' world!');

console.log(buf.toString('hex'));// "416c6f68612c"

console.log(buf.toString('base64'));// "QWxvaGEs"

buf.toString('hex');// "416c6f68612c"

buf.toString('base64');// "QWxvaGEs"

Besides encoding/decoding, there are few more functions for testing[string encoding](#string-types-table).

A JavaScript[String](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String) is a unicode string, which means that it is a[list of unicode characters](https://en.wikipedia.org/wiki/List_of_Unicode_characters), not a list of bytes!

And it does not map one-to-one to an array of bytes without some encoding either.

This is because a unicode character requires 3 bytes to be able to encode any of the growing list of137 000 symbols.

This is because a unicode character requires 3 bytes to be able to encode any of the growing list ofabout 144 000 symbols.

Thus`String` is not the best data type for working with binary data.

This is the main reason why the Node.js devs have come up with the[Buffer](https://nodejs.org/api/buffer.html) type.

Later on there have been invented the[TypedArray](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray) standard to the rescue and the Node.js devs have adopted the new typeas the parent type for the existing`Buffer` type (starting with Node.js v4).

Later on there have been invented the[TypedArray](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray) standard to the rescue and the Node.js devs have adopted the new type, namely`Uint8Array`,as the parent type for the existing`Buffer` type,starting with Node.js v4.

Meanwhile there have been written many libraries to encode, encrypt, hash or otherwise transform the data, all using the plain`String` type that was available to the community since the beginning of JS.

Even some browser built-in functions that came before the`TypedArray` standard rely on the`String` type to do their encoding (eg.[btoa](https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/btoa) == "binary to ASCII").

Today, if you want to manipulate some bytes in JavaScript, you most likely need a`Uint8Array` instead of a`String` for best performance and compatibility with other environments and tools.

Judging by content, there are a few kinds of JS`String`s used in almost all applications.

Most encoding algorithms would not accept a multibyte`String`.

A subset of binary strings is[**ASCII**](https://www.asciitable.com/) only strings,

which represent the class of strings with character codes in the range[0..127].

Each ASCII character can be represented with only 7 bits.

If you try to run`btoa('€')`, you'll get an error like:

constasciiStr='Any text using the 26 English letters, digits and punctuation!';

isASCII(asciiStr);// true

isASCII(binStr);// false

isASCII(utf8Str);// false

UncaughtDOMException:

Failed to execute'btoa' on'Window':

The string to be encoded contains characters outsideof the Latin1 range.

Because`€` is a multibyte character.

The solution is to encode the multibyte string into a singe-byte string somehow.

[UTF8](https://en.wikipedia.org/wiki/UTF-8) is the most used byte encoding of unicode/multibyte strings in computers today. It is the default encoding of web pages that travel over the wire (`content-type: text/html; charset=UTF-8`) and the default in many programing languages.

The important feature of UTF8 is that it is fully compatible with ASCII strings, which means any ASCII string is also a valid UTF8 encoded string. Unless you need symbols outside the ASCII table, this encoding is very compact, and uses more than a byte per character only where needed.

[UTF8](https://en.wikipedia.org/wiki/UTF-8) is the most widely used byte encoding of unicode/multibyte strings in computers today.

It is the default encoding of web pages that travel over the wire (`content-type: text/html; charset=UTF-8`)

and the default in many programing languages.

The important feature of UTF8 is that it is fully compatible with ASCII strings,

which means any ASCII string is also a valid UTF8 encoded string.

Unless you need symbols outside the ASCII table, this encoding is very compact,

and uses more than a byte per character only where needed.

constmbStr='$ ⚔ ₽ 😄 € ™';

str2buffer(utf8Str);// Uint8Array([226, 130, 189, 32, 226, 154, 148, 32, 36, 32, 240, 159, 152, 132, 32, 226, 130, 172, 32, 226, 132, 162])

Even though`utf8Str` is still a`String`, it is no longer a multibyte string.

Even though`utf8Str` is still of type`String`, it is no longer a multibyte string,

and thus can be manipulated as an array of bytes.

A subset of binary strings is[**ASCII**](https://www.asciitable.com/) only strings,

which represent the class of strings with character codes in the range[0..127].

Each ASCII character can be represented with only 7 bits.

constasciiStr='Any text using the 26 English letters, digits and punctuation!';

isASCII(asciiStr);// true

isASCII(binStr);// false

isASCII(utf8Str);// false

All tableheaders are functions exported by this library.

All tableheadings are functions exported by this library.

| String| guessEncoding| hasMultibyte| isBinary| isASCII| isUTF8| utf8bytes|

|:-------------------------:|:-------------:|:------------:|:--------:|:-------:|:------:|:---------:|

| ""|ascii| false| true| true| true| 0|

| ""| hex| false| true| true| true| 0|

| "English alphabet is 26"| ascii| false| true| true| true| 0|

| "$ ⚔ ₽ 😄 € ™"| mb| "⚔"| false| false| false| false|

| utf8Encode("$ ⚔ ₽ 😄 € ™")| utf8| false| true| false| true| 16|

| utf8Decode("Xש")| mb| "Xש"| false| false| false| false|

| "© binary? ×"|~utf8| false| true| false| false| false\| 2|

I did not add the`isHEX` column because it is a trivial format - you can't confuse it with the others.

**Note 1:**

Sometimes you can't tell whether the string has been`utf8Encode`ed