Repository files navigation

Codepages are character encodings. Inmany contexts, single- or double-byte character sets are used in lieu of Unicodeencodings. The codepages map between characters and numbers.

In node:

varcptable=require('codepage');

In the browser:

<scriptsrc="cptable.js"></script><scriptsrc="cputils.js"></script>

Alternatively, use the full version in the dist folder:

<scriptsrc="cptable.full.js"></script>

The complete set of codepages is large due to some Double Byte Character Setencodings. A much smaller file that only includes SBCS codepages is provided inthis repo (sbcs.js), as well as a file for other projects (cpexcel.js)

If you know which codepages you need, you can include individual scripts foreach codepage. The individual files are provided in thebits/ directory.For example, to include only the Mac codepages:

<scriptsrc="bits/10000.js"></script><scriptsrc="bits/10006.js"></script><scriptsrc="bits/10007.js"></script><scriptsrc="bits/10029.js"></script><scriptsrc="bits/10079.js"></script><scriptsrc="bits/10081.js"></script>

All of the browser scripts define and append to thecptable object. To renamethe object, edit theJSVAR shell variable inmake.sh and run the script.

The utilities functions are contained incputils.js, which assumes that theappropriate codepage scripts were loaded.

The script will manipulatemodule.exports if available . This is not alwaysdesirable. To prevent the behavior, defineDO_NOT_EXPORT_CODEPAGE.

Most codepages are indexed by number. To get the Unicode character for a givencodepoint, use thedec property:

varunicode_cp10000_255=cptable[10000].dec[255];// ˇ

To get the codepoint for a given character, use theenc property:

varcp10000_711=cptable[10000].enc[String.fromCharCode(711)];// 255

There are a few utilities that deal with strings and buffers:

var汇总=cptable.utils.decode(936,[0xbb,0xe3,0xd7,0xdc]);varbuf=cptable.utils.encode(936,汇总);varsushi=cptable.utils.decode(65001,[0xf0,0x9f,0x8d,0xa3]);// 🍣varsbuf=cptable.utils.encode(65001,sushi);

cptable.utils.encode(CP, data, ofmt) accepts a String or Array of charactersand returns a representation controlled byofmt:

• Default output is a Buffer (or Array) of bytes (integers between 0 and 255)
• Ifofmt == 'str', return a binary String (bytei iso.charCodeAt(i))
• Ifofmt == 'arr', return an Array of bytes

cptable.utils.decode(CP, data) accepts a byte String or Array of numbers orBuffer and returns a JS string.

A much smaller script, including only the codepages known to be used in Excel,is available under the namecpexcel. It exposes the same variablecptableand is suitable as a drop-in replacement when the full codepage tables are notneeded.

In node:

varcptable=require('codepage/dist/cpexcel.full');

Themake.sh script in the repo can take a manifest and generate JS source.

Usage:

$ bash make.sh path_to_manifest output_file_name JSVAR

where

• JSVAR is the name of the exported variable (generallycptable)
• output_file_name is the output file (cpexcel.js,cptable.js, ...)
• path_to_manifest is the path to the manifest file.

The manifest file is expected to be a CSV with 3 columns:

<codepage number>,<source>,<size>

If a source is specified, it will try to download the specified file and parse.The file format is expected to follow the format from the unicode.org site.The size should be1 for a single-byte codepage and2 for a double-bytecodepage. For mixed codepages (which use some single- and some double-bytecodes), the script assumes the mapping is a prefix code and generates efficientJS code.

Generated scripts only include the mapping.cat a mapping withcputils.jsto produce a complete script likecpexcel.full.js.

This script usesvoc. The script to build the codepage tables andthe JS source iscodepage.md, so building involvesvoc codepage.md.

The complete list of codepages can be found in the filepages.csv.

Some codepages are easier to implement algorithmically. Since those charactertables are not generated, there is no corresponding entry (they are "magic").

unicode.org refers to the Unicode Consortium Public Mappings, a database ofvarious mappings between Unicode characters and respective character sets. Thetables are processed by a few scripts in the build process.

IBM refers to the IBM coded character set database. Even though IBM uses adifferent numbering scheme from Windows, the IBM numbers are used when there isno conflict. The tables are manually generated from the symbol manifests.

Windows 7 refers to direct inspection of Windows 7 machines using .NET classSystem.Text.Encoding. The enclosedMakeEncoding.cs C# program brute-forcescode pages.MakeEncoding.cs deviates from unicode.org in some cases. When theymap a given code to different characters, unicode.org value is used. Whenunicode.org does not prescribe a value,MakeEncoding.cs value is used.

NLS refers to the National Language Support files supplied in various versionsof Windows. In older versions of Windows (like Windows 98) these files followedthe name patternCP_#.NLS, but newer versions use the name patternC_#.NLS.

make test will run the nodejs-based test.

To run the in-browser tests, run a local server and go to thectest directory.make ctestserv will start a pythonSimpleHTTPServer server on port 8000.

To update the browser artifacts, runmake ctest.

• Unicode Consortium Public Mappings
• Windows Code Page Enumeration
• Windows Code Page Identifiers
• IBM Coded Character Sets
• ISO/IEC 2022 / ECMA-35
• International Register of Coded Character Sets To Be Used With Escape Sequences
• Japanese Character Encoding for Internet Messages

Please consult the attached LICENSE file for details. All rights not explicitlygranted by the Apache 2.0 license are reserved by the Original Author.