Unicode® Standard Annex #38

Unicode Han Database (Unihan)

Summary

This document describes the organization and content of the Unihan database.

Status

This document has been reviewed by Unicode members and other interested parties, and has been approved for publication by the Unicode Consortium. This is a stable document and may be used as reference material or cited as a normative reference by other specifications.

A Unicode Standard Annex (UAX) forms an integral part of the Unicode Standard, but is published online as a separate document. The Unicode Standard may require conformance to normative content in a Unicode Standard Annex, if so specified in the Conformance chapter of that version of the Unicode Standard. The version number of a UAX document corresponds to the version of the Unicode Standard of which it forms a part.

Please submit corrigenda and other comments with the online reporting form [Feedback]. Related information that is useful in understanding this annex is found in Unicode Standard Annex #41, “Common References for Unicode Standard Annexes.” For the latest version of the Unicode Standard, see [Unicode]. For a list of current Unicode Technical Reports, see [Reports]. For more information about versions of the Unicode Standard, see [Versions]. For any errata which may apply to this annex, see [Errata].

Contents

• 1 Introduction
• 2 Mechanics
  • 2.1Database Design
    • 2.1.1Extension of Unihan Properties to Non-Unihan Characters
    • 2.1.2Sorting Algorithm Used by the Radical-Stroke Charts
  • 2.2Unihan.zip
  • 2.3Web Access
• 3 Field Types
  • 3.1 IRG Sources
  • 3.2 Other Mappings
  • 3.3 Dictionary Indices
  • 3.4 Readings
  • 3.5 Dictionary-like Data
  • 3.6 Radical-Stroke Counts
  • 3.7 Variants
    • 3.7.1 Simplified and Traditional Chinese Variants
    • 3.7.2 Semantic Variants
  • 3.8 Numeric Values
• 4 The Fields
  • 4.1Alphabetical Listing
  • 4.2Listing by Date of Addition to the Unicode Standard
  • 4.3Listing by Location within Unihan.zip
  • 4.4Listing of Characters Covered by the Unihan Database
• 5History
• References
• Modifications

1Introduction

The Unihan database is the repository for the Unicode Consortium’s collective knowledge regarding the CJK Unified Ideographs contained in the Unicode Standard. It contains mapping data to allow conversion to and from other coded character sets and additional information to help implement support for the various languages which use the Han ideographic script.

Formally, ideographs are defined within the Unicode Standard via their mappings. That is, the Unicode Standard does not formally define what the ideograph U+4E00 is; rather, it defines it as being the equivalent of, say, 0x523B in GB 2312, 0x14421 in CNS 11643, 0x306C in JIS X 0208, and so on.

In practice, implementation of ideographs requires large amounts of ancillary data. Input methods require information such as pronunciations, as do collation algorithms. Data in character sets not included in the world of international standards bodies needs to be converted. Relationships between ideographs need to be defined to allow for fuzzy string matching. Beyond all this, it’s important to track not only what properties a given ideograph has, but who claims it has those properties.

Unlike characters in Western scripts such as Latin and Greek, whose basic property is their sound, which stays largely constant across languages, the basic property for Han ideographs is their meaning. This isn’t to say that ideographs are truly ideographic, in that they represent abstract ideas; but they generally have one root meaning from which the others derive, and generally retain the bulk of their semantic content across linguistic boundaries. Most ideographs are divided into a determinative, which gives a vague sense of meaning, and a phonetic, which gives a vague sense of pronunciation. The Unihan database therefore includes structural analyses and definitions for ideographs.

This document is a guide to that data, describing the mechanics of the Unihan database, the nature of its contents, and the status of the various fields.

2Mechanics

2.1Database Design

The working copy of the Unihan database is maintained privately by the Unicode Consortium. The two public versions are snapshots of this data at a particular point of time.

The database consists of a number of fields containing data for each Han ideograph in the Unicode Standard. The fields are all named, and the names consist entirely of ASCII letters and digits with no spaces or other punctuation except for underscore. For historical reasons, they all start with a lowercase “k.”

Most of these are made available in the public releases. The fields not part of the public releases are, with one exception, either needed only for internal accounting or similar purposes. The remaining private field is a convenience field only; because its value can be determined algorithmically from other data in the database, there is no need to actually include it in the public releases. It is:

• UTF8

  This is the character’s UTF-8 encoding. It is also the only field name not starting with “k”.

All data in the Unihan database is stored in UTF-8 using Normalization Form C (NFC). Note, however, that the "Syntax" descriptions below, used for validation of field values, operate on Normalization Form D (NFD), primarily because that makes the regular expressions simpler.

2.1.1Extension of Unihan Properties to Non-Unihan Characters

Some characters which are not unified ideographs are considered equivalent to unified ideographs. As such, some of the properties defined in this document are applicable to these characters as well, where appropriate.For example,U+2F8D KANGXI RADICAL INSECT is equivalent toU+866B; therefore, properties such askCantonese ("cung4"), orkCangije ("LMI") may be inferred as needed forU+2F8D KANGXI RADICAL INSECT.

This extension process is particularly useful for thekRSUnicode andkTotalStrokes properties.

TheEquivalent_Unified_Ideograph property in the Unicode Character Database is used to indicate which non-ideographs and and unified ideographs are considered equivalent for these purposes. It is explicitly intended to providekRSUnicode andkTotalStrokes values for non-ideographs. See [UAX44] for more information.

2.1.2Sorting Algorithm Used by the Radical-Stroke Charts

The Unicode Standard includes a set of radical-stroke charts for ease in determining the code point of encoded ideographs. Each CJK Unified Ideograph will occur one or more times in the radical-stroke charts, with one occurance per value of itskRSUnicode field in the Unihan Database. Entries in the radical-stroke charts are ordered using a 64-bit collation key calculated as follows:

Bits 0-19 represent the character's code point. This is more space than is actually needed, but it has the advantage of aligning the code point along a four-bit boundary.

Bits 20-27 represent the character's block. This block value is 0 for characters in the CJK Unified Ideographs block, 1 for characters in the CJK Unified Ideographs Extension A, 2 for characters in the CJK Unified Ideographs Extension B block, and so on. The special values 254 (0xFE) and 255 (0xFF) are used for characters in the CJK Compatibility Ideographs and CJK Compatibility Ideographs Supplement blocks, respectively. This allows accomodation for future CJK Unified Ideograph Extension blocks and guarantees that compatibility ideographs always follow non-compatibility ideographs. Note that additional compatibility ideograph blocks will not be encoded in the future.

Bits 28-31 are used to indicate whether the entry has a simplified form for the radical or not. The value of 1 indicates the simplified form of the radical (e.g., 钅); a value of 0 indicates the traditional form for the radical (e.g., 金).

Bits 32-35 are reserved to hold the entry's first residual stroke, as defined by the IRG. This data is currently unavailable and so these bits are always 0.

Bits 36-43 are used for the entry's residual stroke count. If the residual stroke count is negative, 0 is substituted.

Bits 44-51 are used for the entry's KangXi radical.

Bits 52-63 are unused.

This collation key is defined in such a fashion that it can easily be parsed by eye. Figure 1 illustrates its overall structure.

Figure 1.Radical-Stroke Chart Collation Key Schema

Examples:

• U+4E95 (井) is assigned the collation key0x0000702000004E95
• U+3687 (㚇) has two values in itskRSUnicode field and therefore two entries in the radical-stroke charts. These two entries are assigned the collation keys0x0002306000103687 and0x0004206000103687
• U+F936 (虜) is assigned the collation key0x0001C1000FE0F936
• U+21FEB (𡿫) is assigned the collation key0x0002F00000221FEB
• U+2B50E (𫔎) is assigned the collation key0x000A70C01032B50E

2.2Unihan.zip

Included with the Unicode Character Database is a file calledUnihan.zip. This is a snapshot of the public contents of the Unihan database as of the release date for this version of the standard.

The zip file is an archive of eight text files, each in UTF-8, NFC, and using Unix line endings. Each file contains the values for some of the fields in the Unihan database.

Each file contains those properties which belong to one of the general categories described below; that is, Readings.txt contains all data for all the fields in the Readings category, and so on.

Each file uses the same structure. Blank lines may be ignored. Lines beginning with # are comment lines used to provide the header and footer. Each of the remaining lines is one entry, with three, tab-separated fields: the Unicode Scalar Value, the database field name, and the value for the database field for the given Unicode Scalar Value. For most of the fields, if multiple values are possible, the values are separated by spaces. No character may have more than one instance of a given field associated with it, and no empty fields are included in any of the files archived insideUnihan.zip.

There is no formal limit on the lengths of any of the field values. Any Unicode characters may be used in the field values except for double quotes and control characters (especially tab, newline, and carriage return). Most fields have a more restricted syntax, such as thekKangXi field which consists of multiple, space-separated entries, with each entry consisting of four digits 0 through 9, followed by a period, followed by three more digits.

The data lines are sorted by Unicode Scalar Value and field-type as primary and secondary keys, respectively.

Each file’s header includes a summary of the fields the file contains.

2.3Web Access

The URI for interactive access to the contents of the Unihan database ishttp://www.unicode.org/charts/unihan.html. For production reasons, the version available for interactive access may not be immediately updated to the latest available version of the Unihan.zip file.

Links to Chinese and Japanese compound data are presented with this Web front end such as to the onlineCEDICT andJim Breen’s EDICT projects. These additional data are not available in the other versions.

There are also two indices: a grid index grouping the characters in blocks of 256 and a radical-stroke index. A search page is also available. Individual characters can be accessed through the index or via the “Lookup” button and text field above. You enter the four- or five-digit hexadecimal identifier for the character, and click “Lookup”. You will be taken to an information page for the character. The “Use text, not images” check-box allows you to control whether UTF-8 text or embedded GIFs will be used in to display ideographs. The latter technique is less dependent on your browser and system support for Unicode but is much slower.

3Field Types

The data in the Unihan database serves a multitude of purposes, and the fields are most conveniently grouped into categories according to the purpose they fulfil. We provide here a general discussion of the various categories, followed by a detailed description of the individual fields, alphabetically arranged.

Again, it is important to remember that all data in the Unihan database has been donated to the Unicode Consortium. Unicode currently has no staff with the responsibility to maintain or update the Unihan database. This means that, for example, the data is more complete for Chinese than for other languages simply because more data has been donated for Chinese than for other languages.

3.1IRG Sources

Among the few normative parts of the Unihan database, and the most exhaustively checked fields, are the nine IRG source fields:kIRG_GSource (PRC and Singapore),kIRG_HSource (Hong Kong SAR),kIRG_JSource (Japan),kIRG_KPSource (North Korea),kIRG_KSource (South Korea),kIRG_MSource (Macao),kIRG_TSource (Taiwan),kIRG_USource (Unicode/USA), andkIRG_VSource (Vietnam).

These represent the official mappings between Unihan and the various encoded character sets or collections which have been submitted by IRG members. The versions of these standards may differ from the published versions generally available, particularly for PRC standards. This is because in the early days of Unicode, the PRC would occasionally add characters to their standards on an ad hoc basis in order to make sure they were included. The various procedures involved in submitting characters to the IRG for consideration no longer make this necessary.

The values for the U-source were, in the past, only references to the Unicode Standard itself and were always equal to the character’s Unicode Scalar Value. This changed with the inclusion of Extension C in version 5.2.0 of the Unicode Standard. The values now include indices as described in [UAX45].

The syntax for the values used in the various IRG source fields matches that found in ISO/IEC 10646:2011.

Detailed descriptions of the syntax used are to be found inSection 4.1Alphabetical Listing below.

Note that we do not include the four IRG dictionary fields in this category, largely because they are not normative parts of the standard.

ThekIICore field is also defined by the IRG and normative.

3.2Other Mappings

There are twenty-four fields in this category. They consist of mapping tables between the ideographic portions of Unicode and those of encoded character sets or character collectionsnot used by the IRG in its work, although some of the character sets covered do mirror official IRG sources. For example, data for mapping GB 12345 is included, even though GB 12345 is a part of the IRG’s G-source. The difference between the two is that thekGB1 field maps all of GB 12345 to Unicode, and not just that portion included in the G-source, and it doesn’t map any of the informal extensions to GB 12345.

3.3Dictionary Indices

There are three main reasons for providing indices into standard dictionaries.

First, standard dictionaries provide a “paper trail” for fields such as the English gloss (kDefinition) and the various pronunciations or readings, as well as variant data.

Second, standard dictionaries provide a reference for scholars or students who wish more information about a character.

Third, standard dictionaries are a source for unencoded characters. This is particularly important for Cantonese, where the Cantonese lexicon is not standardized and has been neglected by the authors and architects of previous character set encodings other than HK SCS.

As elsewhere, the set of dictionaries covered represent data that has been volunteered. There are important dictionaries (for example, theHanyu Da Cidian, theShuowen) for which formal indices should be provided. And as elsewhere, the data which has been volunteered is weighted heavily in favor of Chinese.

Four of the dictionary fields represent official IRG indices for the dictionaries used in the four dictionary sorting algorithm. Two (kIRGHanyuDaZidian andkIRGKangXi) are still being used by the IRG, but the other two (kIRGDaeJaweon andkIRGDaiKanwaZiten) are not. We have, nonetheless, retained their data for reference purposes.

For all four, there are clone fields to hold Unicode indices into the same four dictionaries. By and large, the data in the IRG fields and their Unicode counterparts is the same—but not always.

The remaining dictionaries can be grouped into three categories: general-purpose Chinese (including classical Chinese and Mandarin), Cantonese, and other.

The general-purpose Chinese dictionary fields are:kCihaiT,kFennIndex,kGSR,kKarlgren,kMatthews, andkSBGY. These represent large, standard Chinese-Chinese, Chinese-English dictionaries, or definitive sinological studies.

The Cantonese dictionary fields arekCheungBauerIndex,kCowles,kLau, andkMeyerWempe. All but Cheung-Bauer are large character-based Cantonese-English dictionaries.

At present, the only other dictionary field iskNelson, the character’s index in the first edition of Andrew N. Nelson’s excellent and popularModern Reader’s Japanese-English Character Dictionary.

In selecting dictionaries for inclusion—outside of the general consideration of who is willing to volunteer what data—we aim for including large dictionaries rather than small ones, and standard dictionaries such as serious students might have on their shelves.

3.4Readings

We include in this category the pronunciations for a given character in Mandarin, Cantonese, Tang-dynasty Chinese, Japanese, Sino-Japanese, Korean, and Vietnamese. We also include here the English gloss for a given character.

Any attempt at providing a reading or set of readings for a character is bound to be fraught with difficulty, because the readings will vary over time and from place to place, even within a language. Mandarin is the official language of both the PRC and Taiwan (with some differences between the two) and is the primary language over much of northern and central China, with vast differences from place to place. Even Cantonese, the modern language covered by the Unihan database with the least geographical range, is spoken throughout Guangdong Province and in much of neighboring Guangxi, and covers four large urban centers (Guangzhou, Shenzhen, Macao, and Hong Kong), with Guangzhou Cantonese somewhat infected by Mandarin and Hong Kong Cantonese more than a little infected by English.

Indeed, even the same speaker will pronounce the same word differently depending on the speaker or even the social context. This is particularly true for languages such as Cantonese, where there has been comparatively little government effort to standardize the language.

Add to this the fact that in none of these languages—the various forms of Chinese, Japanese, Korean, Vietnamese—is the syllable the fundamental unit of the language. As in the West, it’s the word, and the pronunciation of a character is tied to the word of which it is a part. In Chinese (followed by Vietnamese and Korean), the rule is one ideograph/one syllable, with most words written using multiple ideographs. In most cases, an ideograph has only one reading (or only one important reading), but there are numerous exceptions.

In Japanese, the situation is enormously more complex. Japanese has two pronunciation systems, one derived from Chinese (theon pronunciation, or Sino-Japanese), and the other from Japanese (thekun pronunciation).

Theon readings derive from Chinese loan-words. They depend on factors such as when (and from which part of China) the loan-word was borrowed, and changes to Japanese since then.On readings can therefore have little obvious relationship to modern Chinese readings, and the same Chinese reading for a givenkanji can be reflected in multipleon readings in Japanese. Contrary to Chinese practice,on readings may be polysyllabic.

Kun readings, on the other hand, derive from native Japanese words for which either existingkanji were adopted or newkanji coined.

The net result is that multiple readings are the rule for Japanesekanji. These multiple readings may bear no relationship to one another and are highly context-sensitive. Even a native Japanese reader may not know the correct pronunciation of a proper noun if it is written only inkanji.

Finally, some characters have rare pronunciations known only to a minority of native speakers, or are so rare themselves that few, if any, native speakers know how to pronounce them (for example, U+40DF 䃟, used in a Hong Kong place name). In many cases, the pronunciations given by professional lexicographers are little more than educated guesses.

Thus, unlike mappings between Unicode and other character sets, providing definitive data on pronunciations or, similarly, providing a definitive English gloss is impossible, and not something which has been achieved. While we make every effort to use our sources judiciously, we are aware of the fact that this data can always be improved and extended. Users should not naïvely assume that learning to pronounce an East Asian language is all about learning to pronounce the individual ideographs, or that reading is done by parsing the ideographs, one at a time.

Despite these caveats, the reading and definition data is very useful both for the student attempting to learn these languages, and for the professional attempting to use them, and so the data is included in the Unihan database.

3.5Dictionary-like Data

This category is something of a hodge-podge, consisting of various fields including information one might find in a dictionary (such as a character’scangjie input code), or data useful in determining levels of support (such as frequency), or structural analyses which can be helpful in lookup systems (such as the character’s phonetic).

As with the readings and English gloss, this data does not cover as much of Unihan as is theoretically possible, although it does cover the bulk of what is used day-to-day.

3.6Radical-Stroke Counts

We include six radical-stroke counts for Unihan, although only three (kRSAdobe_Japan1_6,kRSKangXi, andkRSUnicode) can be considered complete; the others (kRSJapanese,kRSKanWa, andkRSKorean) are placeholders to be filled in later. Three are based on IRG standard dictionaries: theHanyu Da Zidian, which uses a slightly different radical system from the others, is not included, althoughHanyu Da Zidian radical-stroke data can be calculated using thekHDZRadBreak field.

All the radical-stroke fields are based on the radical-system introduced by the 18th-centuryKangXi dictionary. Each ideograph is assigned one of 214 radicals. In most cases, the radical assigned is the natural radical, giving a clue as to the character’s meaning; in the rest, the radical is arbitrary, based on the character’s structure. One also counts the character’s residual strokes, that is, the number of brush strokes required to write everything in the character except the radical.

To find a character using the radical-stroke system, one determines its radical and the number of residual strokes, then looks through the list of characters with those characteristics. This is a clumsy system compared to alphabetical lookup, but is one of the most widespread systems throughout East Asia. Unfortunately, it is also ambiguous.

First of all, if a character does not have a natural radical, it can sometimes be hard to tell what the radical ought to be (for example, 井 being assigned arbitrarily the radical 二). Even if the character naturally falls into radical-like pieces, it can be hard to tell which is the radical and which the phonetic (for example, 和, which looks like it belongs to the radical 禾, actually belongs to the radical 口). Moreover, since Unicode encodes characters, not glyphs, two different glyphs for the same character may have different residual strokes (such as 者, which can be written either with or without a dot, altering its stroke count between nine and eight, respectively).

We include multiple radical-stroke systems to allow for this. Three of the radical-stroke fields represent the character’s radical-stroke count as determined by its position within a standard IRG dictionary. Two more (kRSJapanese andkRSUnicode) are intended to cover a “typical” Japanese radical-stroke count, and everything else, respectively. Finally, there is thekRSAdobe_Japan1_6 field which contains more detailed information on the glyph used for the character in the Adobe Japan 1-6 character set.

The primary use for thekRSUnicode field is to cover the normative radical-stroke value defined by ISO/IEC 10646. However, it is also used for cases where there is sufficient ambiguity that a reasonable person might look for a character in multiple places, particularly where one of our source dictionaries categorizes a character under a different radical or with a different stroke count.

ThekRSUnicode field also uses an apostrophe after the radical number to indicate that the character uses a standard simplification. In simplified Chinese, many radicals have standard, simplified forms, such as 讠, which is the simplified form of the radical 言

There is, by the way, no standard way of ordering characters within a given radical-stroke group. Unicode’s radical-stroke charts order characters with the same radical-stroke count by the Unicode block in which they occur. If looking for a character with radical 64 (手) and ten residual strokes, one knows that of the 175 candidates in Unicode 5.2.0, the most common ones come towards the head of the list and the less common ones later.

The IRG is in the process of adopting a common system of assigning the first stroke of the phonetic element to one of five categories, and sorting by those categories. When this “first stroke” data is available for all of Unihan, it will be added to the Unihan database and simplify the process of finding a character within a particular radical-stroke block.

3.7Variants

Although Unicode encodes characters and not glyphs, the line between the two can sometimes be hard to draw, particularly in East Asia. There, thousands of years worth of writing have produced thousands of pairs which can be used more-or-less interchangeably.

To deal with this situation, the Unicode Standard has adopted a three-dimensional model for determining the relationship between ideographs, and has formal rules for when two forms may be unified. Both are described in some detail in the Unicode Standard. Briefly, however, the three-dimensional model uses the x-axis to represent meaning, and the y-axis to represent abstract shape. The z-axis is used for stylistic variations.

To illustrate, 說 and 貓 have different positions along the x-axis, because they mean two entirely different things (to speak andcat, respectively). 貓 and 猫 mean the same thing and are pronounced the same way but have different abstract shapes, so they have the same position on the x-axis (semantics) but different positions on the y-axis (abstract shape). They are said to be y-variants of one another. On the other hand, 說 and 説 have the same meaning and pronunciation and the same abstract shape, and so have the same positions on both the x- and y-axes but different positions on the z-axis. They are z-variants of one another.

Ideally, there would be no pairs of z-variants in the Unicode Standard; however, the need to provide for round-trip compatibility with earlier standards, and some out-and-out mistakes along the way, mean that there are some. These are marked using thekZVariant field.

The remaining variant fields are used to mark different types of y-variation.

3.7.1Simplified and Traditional Chinese Variants

ThekTraditionalVariant andkSimplifiedVariant fields are used in character-by-character conversions between simplified and traditional Chinese (SC and TC, respectively). For any character X, when converting between SC and TC, there are four possible cases:

1. X is used in both SC and TC and is unchanged when mapping between them. An example would be 井 U+4E95. This is the most common case, and is indicated by both thekSimplifiedVariant andkTraditionalVariant fields being empty.
2. X is used in TC but not SC, that is, it is changed when converting from TC to SC, but not vice versa. In this case, thekSimplifiedVariant field lists the character(s) to which it is mapped and thekTraditionalVariant field is empty. An example would be 書 U+66F8 whosekSimplifiedVariant field is 书 U+4E66.
3. X is used in SC but not TC, that is, it is changed when converting from SC to TC, but not vice versa. In this case, thekTraditionalVariant field lists the character(s) to which it is mapped and thekSimplifiedVariant field is empty. An example would be 学 U+5B66 whosekTraditionalVariant field is 學 U+5B78.
4. X is used in both SC and TC and may be changed when mapping between them. This is the most complex case, because there are two distinct sub-cases:
   1. X may be mapped to itself or to another character when converting between SC and TC. In this case, the character is its own simplification as well as the simplification for other characters. An example would be 后 U+540E, which is the simplification for itself and for 後 U+5F8C. When mapping TC to SC, it is left alone, but when mapping SC to TC it may or may not be changed, depending on context. In this case, bothkTraditionalVariant andkSimplifiedVariant fields are defined and X is included among the values for both.
   2. X is used for different words in SC and TC. When converting between the two, it is always changed. An example would be 苧 U+82E7. In traditional Chinese, it is pronounced zhù and refers to a kind of nettle. In simplified Chinese, it is pronounced níng and means limonene (a chemical found in the rinds of lemons and other citrus fruits). When converting TC to SC it is mapped to 苎 U+82CE, and when converting SC to TC it is mapped to 薴 U+85B4. In this case, bothkTraditionalVariant andkSimplifiedVariant fields are defined but X is not included in the values for either.

In practice, conversion between simplified and traditional Chinese is complicated by three factors:

1. The conversion is almost always one-to-one, but in some cases may be one-to-many, and context may need to be evaluated to determine which specific mapping to use. When converting SC to TC, 脏 U+810F is mapped to 臟 U+81DF when it means "viscera" and to 髒 U+9AD2 when it means "dirty."
2. An SC character may be used in actual TC text and, more rarely, vice versa. This is particularly true in handwritten and ancient texts. Indeed, many SC forms originated as handwritten forms or ancient synonyms. It also occurs when one of a number of synonymous TC characters is identified as the preferred or correct character to use in SC. For example, both 猫 U+732B and 貓 U+8C93 are acceptable TC characters meaning "cat," but only 猫 U+732B should be used in SC.
3. Political divisions within the Chinese-speaking community have resulted in different coinages in different locales for various modern terms, and so actual conversion between SC and TC is ideally done on a word-by-word basis, not a character-by-character basis. A hard disk, for example, is called 硬盘 in the PRC, and 硬碟 in Taiwan.

3.7.2Semantic Variants

The remaining two variation fields,kSemanticVariant andkSpecializedSemanticVariant, are used to mark cases where two characters have identical and overlapping meanings, respectively.

Thus U+514E 兎 and U+5154 兔 are y-variants of one another; both meanrabbit. U+4E3C 丼 and U+4E95 井 are not pure y-variants of one another. 井 meansa well, and although 丼 can also meana well and be used for 井, it can also meana bowl of food. We usekSemanticVariant, then, for the former pair, andkSpecializedSemanticVariant for the latter. In many cases, data is provided listing the Unihan sources which indicate the variant relationship. The syntax is described in detail below, but as an example, U+792E 礮 has thekSemanticVariant valueU+70AE<kMeyerWempe U+7832<kLau,kMatthews,kMeyerWempe U+791F<kLau,kMatthews. This means that the Mathews, Lau, and Meyer-Wempe dictionaries all say that it is a y-variant of U+7832 砲, whereas only Mathews and Lau identify it as a variant of U+791F 礟 and only Meyer-Wempe identifies it as a variant of U+70AE 炮.

3.8Numeric Values

Finally, we have three fields,kAccountingNumeric,kOtherNumeric, andkPrimaryNumeric to indicate the numerical values an ideograph may have. Traditionally, ideographs were used both for numbers and words, and so many ideographs have (or can have) numeric values. The various kinds of numeric values are specified by these three fields.

4The Fields

We now give two listings of the fields in the Unihan database. The first is an alphabetical listing, with information on the field contents and syntax. The second is a listing of the fields by the release of the Unicode Standard in which they were first found.

4.1Alphabetical Listing

For each field we give the following information in the alphabetical listing: itsProperty tag, its UnicodeStatus, itsCategory as defined above, the Unicode version in which it wasIntroduced, itsDelimiter, itsSyntax, and itsDescription.

TheProperty name is the tag used in the Unihan database to mark instances of this field.

The UnicodeStatus is eitherNormative,Informative, orProvisional, depending on whether it is a normative part of the standard, an informative part of the standard, or neither. We may also includeDeprecated as a Unicode Status if the field is no longer to be used.

Fields which allow multiple values have aDelimiter defined as “space”. Fields which do not have multiple values (such as the IRG source fields) have this defined as “N/A”. Some fields do not currently have multiple values in the data but may do so in the future.

For most fields with multiple values, the order of the values is arbitrary and has no particular significance. The most common order in such cases is alphabetical. For example, see the kCantonese field.

However, for certain fields the ordering of values may be significant; in such cases, the significance is specified in the Description for the field. For example, see the kMandarin field. In later versions of the Unicode Character Database, a field may change from arbitrary order to a specified order.

Validation is done as follows: The entry is split into subentries using theDelimiter (if defined), and each subentry converted to Normalization Form D (NFD). The value is valid if and only if each normalized subentry matches the field’sSyntax regular expression. Note that the value for any given field'sSyntax is not guaranteed to be stable and may change in the future.

Finally, theDescription contains not only a description of what the field contains, but also source information, known limitations, methodology used in deriving the data, and so on.

The fields covered in the table are:kAccountingNumeric,kBigFive,kCangjie,kCantonese,kCCCII,kCheungBauer,kCheungBauerIndex,kCihaiT,kCNS1986,kCNS1992,kCompatibilityVariant,kCowles,kDaeJaweon,kDefinition,kEACC,kFenn,kFennIndex,kFourCornerCode,kFrequency,kGB0,kGB1,kGB3,kGB5,kGB7,kGB8,kGradeLevel,kGSR,kHangul,kHanYu,kHanyuPinlu,kHanyuPinyin,kHDZRadBreak,kHKGlyph,kHKSCS,kIBMJapan,kIICore,kIRG_GSource,kIRG_HSource,kIRG_JSource,kIRG_KPSource,kIRG_KSource,kIRG_MSource,kIRG_TSource,kIRG_USource,kIRG_VSource,kIRGDaeJaweon,kIRGDaiKanwaZiten,kIRGHanyuDaZidian,kIRGKangXi,kJa,kJapaneseKun,kJapaneseOn,kJinmeiyoKanji,kJis0,kJis1,kJIS0213,kJoyoKanji,kKangXi,kKarlgren,kKorean,kKoreanEducationHanja,kKoreanName,kKPS0,kKPS1,kKSC0,kKSC1,kLau,kMainlandTelegraph,kMandarin,kMatthews,kMeyerWempe,kMorohashi,kNelson,kOtherNumeric,kPhonetic,kPrimaryNumeric,kPseudoGB1,kRSAdobe_Japan1_6,kRSJapanese,kRSKangXi,kRSKanWa,kRSKorean,kRSUnicode,kSBGY,kSemanticVariant,kSimplifiedVariant,kSpecializedSemanticVariant,kTaiwanTelegraph,kTang,kTGH,kTotalStrokes,kTraditionalVariant,kVietnamese,kXerox,kXHC1983,andkZVariant.

4.2Listing by Date of Addition to the Unicode Standard

The table below lists the fields of the Unihan database by the release where they were first added. Also included are fields which were dropped in a particular release. These are indicated by italics.

The remaining fields were added prior to Unicode 2.0.

4.3Listing by Location within Unihan.zip

The table below lists the fields of the Unihan database. They are organized into groups according to the file within Unihan.zip where their values are found. Each field name also links to its description.

4.4Listing of Characters Covered by the Unihan Database

The following table lists the characters covered by the Unihan database, together with the version in which they were added to the Unicode Standard.

Note that some CJK charactersdo not explicitly have property data in the Unihan database, such as:

5History

The Unihan database originated as a Hypercard stack using data provided by such organizations as Apple, RLG, and Xerox. Printed versions are found inThe Unicode Standard, Version 1.0, volume 2. Electronic versions were available on floppy disk in the form of a file called CJKXREF.TXT.

The first general electronic release ofCJKXREF.TXT (961 kB) was included with Unicode 1.1.5 in July 1995. This version of the file is in a multi-column format and includes the data used in printingThe Unicode Standard, Version 1.0, volume 2 with the exception of the Fujitsu mappings, which were found to be incorrect and withdrawn.

The electronic version of the Unihan database was substantially revised for the publication of Unicode 2.0.0 in July 1996. The file was renamed UNIHAN.TXT; its permanent, archival link isUnihan-1.txt (7.9 MB). The format of the file is essentially the same as the current release, although consolidated into a single file. The fields were explicitly named for the first time. The data was at the time maintained using custom, MacApp-based database software. The source code for this software used an enumerated type for the numeric field tags, and the enumerator names (each beginning with a "k" indicating their use as a constant) were used in the text file as field names.

Unihan-1.txt was at some point accidentally truncated on line 330,553 (partway through the data for U+8BC1). No corrected version of the file was made available. Instead, it was superseded by theUnihan-2.txt (10 MB) file released with Unicode 2.1.2 in May 1998.

The difficulty of downloading a file 19 MB in size with the technology of the time led to the Unihan database being made available as both a single text file and compressed archives of that text file as of Unicode 3.1.0 in March 2001. The format of the Unihan database remained essentially unchanged until Unicode 5.1.0 (April 2008), when the text file was no longer included and the database became available only as a zipped archive.

Finally, the archive was changed from containing one text file to containing multiple text files as of Unicode 5.2.0 (October 2009).

References

For references for this annex, see Unicode Standard Annex #41, “Common References for Unicode Standard Annexes.”

Modifications

The following summarizes modifications from the previous revision of this annex.

Revision 27

• Reissued for Unicode 12.0.0.
• Removed discussion ofkDefaultSortKey and added description of the sorting algorithm used for radical-stroke charts.
• Updated syntax and description forkIRG_GSource.
• Updated syntax forkIRG_JSource andkIRG_MSource.
• Updated description and syntax for forkIRG_KSource,kIRG_TSource, andkIRG_USource.

Revision 26 being a proposed update, only changes between revisions 25 and 27 are noted here.

Modifications for previous versions are listed in those respective versions.

© 2019 Unicode, Inc. All Rights Reserved. The Unicode Consortium makes no expressed or implied warranty of any kind, and assumes no liability for errors or omissions. No liability is assumed for incidental and consequential damages in connection with or arising out of the use of the information or programs contained or accompanying this technical report. The UnicodeTerms of Use apply.

Unicode and the Unicode logo are trademarks of Unicode, Inc., and are registered in some jurisdictions.