Main features

Installation

pip install chirptext

Parsing Japanese text

>>>fromchirptextimportdeko>>>sent=deko.parse('猫が好きです。')>>>sent.tokens['`猫`<0:1>','`が`<1:2>','`好き`<2:4>','`です`<4:6>','`。`<6:7>']>>>sent.tokens.values()['猫','が','好き','です','。']>>>sent[0]`猫`<0:1>>>>sent[0].pos'名詞'>>>sent[1].lemma'が'>>>sent[2].reading'スキ'# tokenize>>>deko.tokenize('猫が好きです。')['猫','が','好き','です','。']# split sentences>>>deko.tokenize_sent("猫が好きです。\n犬も好きです。")['猫が好きです。','犬も好きです。']# parse a document (i.e. multiple sentences)>>>doc=deko.parse_doc("猫が好きです。\n犬も好きです。")>>>forsentindoc:...print(sent,sent.tokens.values())...猫が好きです。 ['猫','が','好き','です','。']犬も好きです。 ['犬','も','好き','です','。']

Convenient IO APIs

>>>fromchirptextimportchio>>>chio.write_tsv('data/test.tsv', [['a','b'], ['c','d']])>>>chio.read_tsv('data/tes.tsv')[['a','b'], ['c','d']]>>>chio.write_file('data/content.tar.gz','Support writing to .tar.gz file')>>>chio.read_file('data/content.tar.gz')'Support writing to .tar.gz file'>>>forrowinchio.read_tsv_iter('data/test.tsv'):...print(row)... ['a','b']['c','d']

Sample TextReport

# a string reportrp=TextReport()# by default, TextReport will write to standard output, i.e. terminalrp=TextReport(TextReport.STDOUT)# same as aboverp=TextReport('~/tmp/my-report.txt')# output to a filerp=TextReport.null()# ouptut to /dev/null, i.e. nowhererp=TextReport.string()# output to a string. Call rp.content() to get the stringrp=TextReport(TextReport.STRINGIO)# same as above# TextReport will close the output stream automatically by using the with statementwithTextReport.string()asrp:rp.header("Lorem Ipsum Analysis",level="h0")rp.header("Raw",level="h1")rp.print(LOREM_IPSUM)rp.header("Top 5 most common letters")ct.summarise(report=rp,limit=5)print(rp.content())

Output

+---------------------------------------------------------------------------------- | Lorem Ipsum Analysis +----------------------------------------------------------------------------------  Raw ------------------------------------------------------------ Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.  Top 5 most common letters------------------------------------------------------------ i: 42 e: 37 t: 32 o: 29 a: 29

Useful links