Kagome v2

Note

Key features (Improvements fromv1):

• Self-contained binaries with embedded dictionaries (MeCab-IPADIC, UniDic)
• Multiple segmentation modes for different use cases
• RESTful API server mode for production use
• WebAssembly support for browser environments

Index

Basic Usage

Command line

%kagome -hJapanese Morphological Analyzer -- github.com/ikawaha/kagome/v2usage: kagome <command>The commands are:   [tokenize] - command line tokenize (*default)   server - run tokenize server   lattice - lattice viewer   sentence - tiny sentence splitter   version - show versiontokenize [-file input_file] [-dict dic_file] [-userdict user_dic_file] [-sysdict (ipa|uni)] [-simple false] [-mode (normal|search|extended)] [-split] [-json]  -dict string    dict  -file string    input file  -json    outputs in JSON format  -mode string    tokenize mode (normal|search|extended) (default "normal")  -simple    display abbreviated dictionary contents  -split    use tiny sentence splitter  -sysdict string    system dict type (ipa|uni) (default "ipa")  -udict string    user dict

%# piped standard input%echo"すもももももももものうち"| kagomeすもも名詞,一般,*,*,*,*,すもも,スモモ,スモモも助詞,係助詞,*,*,*,*,も,モ,モもも名詞,一般,*,*,*,*,もも,モモ,モモも助詞,係助詞,*,*,*,*,も,モ,モもも名詞,一般,*,*,*,*,もも,モモ,モモの助詞,連体化,*,*,*,*,の,ノ,ノうち名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチEOS

As a Go library

# Install Kagome modulego get github.com/ikawaha/kagome/v2

package mainimport ("fmt""strings""github.com/ikawaha/kagome-dict/ipa""github.com/ikawaha/kagome/v2/tokenizer")funcmain() {t,err:=tokenizer.New(ipa.Dict(),tokenizer.OmitBosEos())iferr!=nil {panic(err)  }// wakati (simple word splitting/segmentation)fmt.Println("---wakati---")seg:=t.Wakati("すもももももももものうち")fmt.Println(seg)// tokenize w/ morphological analysisfmt.Println("---tokenize---")tokens:=t.Tokenize("すもももももももものうち")for_,token:=rangetokens {features:=strings.Join(token.Features(),",")fmt.Printf("%s\t%v\n",token.Surface,features)  }}

---wakati---[すもも も もも も もも の うち]---tokenize---すもも名詞,一般,*,*,*,*,すもも,スモモ,スモモも助詞,係助詞,*,*,*,*,も,モ,モもも名詞,一般,*,*,*,*,もも,モモ,モモも助詞,係助詞,*,*,*,*,も,モ,モもも名詞,一般,*,*,*,*,もも,モモ,モモの助詞,連体化,*,*,*,*,の,ノ,ノうち名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ

Install

Commands

Tokenize command

%# interactive/REPL mode%kagomeすもももももももものうちすもも名詞,一般,*,*,*,*,すもも,スモモ,スモモも助詞,係助詞,*,*,*,*,も,モ,モもも名詞,一般,*,*,*,*,もも,モモ,モモも助詞,係助詞,*,*,*,*,も,モ,モもも名詞,一般,*,*,*,*,もも,モモ,モモの助詞,連体化,*,*,*,*,の,ノ,ノうち名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチEOS

%# piped standard inputecho "すもももももももものうち" | kagomeすもも  名詞,一般,*,*,*,*,すもも,スモモ,スモモも      助詞,係助詞,*,*,*,*,も,モ,モもも    名詞,一般,*,*,*,*,もも,モモ,モモも      助詞,係助詞,*,*,*,*,も,モ,モもも    名詞,一般,*,*,*,*,もも,モモ,モモの      助詞,連体化,*,*,*,*,の,ノ,ノうち    名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチEOS

%# JSON output%# (For jq command see https://jqlang.org/)%echo"猫"| kagome -json| jq.[  {    "id": 286994,    "start": 0,    "end": 1,    "surface": "猫",    "class": "KNOWN",    "pos": [      "名詞",      "一般",      "*",      "*"    ],    "base_form": "猫",    "reading": "ネコ",    "pronunciation": "ネコ",    "features": [      "名詞",      "一般",      "*",      "*",      "*",      "*",      "猫",      "ネコ",      "ネコ"    ]  }]

%# word splitting/segmentation only (equivalent to "wakati" functionality)%echo"すもももももももものうち"| kagome -json| jq -r'[.[].surface] | join("/")'すもも/も/もも/も/もも/の/うち

%# Extract only pronunciations using jq (for Text-to-Speech purposes, etc.)%echo"私ははにわよわわわんわん"| kagome -json| jq -r'.[].pronunciation'ワタシワハニワヨワワワンワン

Server command

RESTful API

%kagome server&%curl -XPUT localhost:6060/tokenize -d'{"sentence":"すもももももももものうち", "mode":"normal"}'| jq.

Web App

%kagome server&

Important

The demo web application usesgraphviz to draw a lattice. You need graphviz to be installed on your system.

[!TIP]Kagome can be compiled to WebAssembly (wasm) and run locally in a web browser as well. For details, see theWebAssembly section.

• Wasm Demo:https://ikawaha.github.io/kagome/

Lattice command

%kagome lattice 私は鰻| dot -Tpng -o lattice.png

Sentence command

%echo"吾輩は猫である。名前はまだ無い。"| kagome sentence吾輩は猫である。名前はまだ無い。

%echo"吾輩は猫である。名前はまだ無い。"| kagome -json| jq -r'[.[].surface] | join("/")'吾輩/は/猫/で/ある/。/名前/は/まだ/無い/。%echo"吾輩は猫である。名前はまだ無い。"| kagome sentence| kagome -json| jq -r'[.[].surface] | join("/")'吾輩/は/猫/で/ある/。名前/は/まだ/無い/。

%echo"吾輩は猫である。名前はまだ無い。"| kagome -split -json| jq -r'[.[].surface] | join("/")'吾輩/は/猫/で/ある/。名前/は/まだ/無い/。

Dictionaries

Note

For more details and differences between the dictionaries, see thewiki.

Segmentation modes

Note

If your purpose is for search, try changing the mode before switching to another dictionary.

Docker

WebAssembly

Reference

License