Table of contents

1. Naver ClovaCall dataset contribution

The dataset statistics

The dataset structure

ClovaCall.json[  {    "wav" : "42_0603_748_0_03319_00.wav",    "text" : "단체 할인이 가능한 시간대가 따로 있나요?",    "speaker_id" : "03319"  },  ...,  {    "wav" : "42_0610_778_0_03607_01.wav",    "text" : "애기들이 놀만한 놀이방이 따로 있나요?",    "speaker_id" : "03607"  }]

2. Dataset downloading and license

To all the materials including speech data distributed here(hereinafter, “MATERIALS”), the following license(hereinafter, “LICENSE”) shall apply. If there is any conflict between the LICENSE and the clovaai/speech_hackathon_19 License(Apache Lincese 2.0) listed on Github, the LICENSE below shall prevail.1. You are allowed to use the MATERIALS ONLY FOR NON-COMMERCIAL AI(Artificial Intelligence) RESEARCH AND DEVELOPMENT PURPOSES – ANY KIND OF COMMERCIAL USE IS STRICTLY PROHIBITED.2. You should USE THE MATERIALS AS THEY WERE PROVIDED – ANY KIND OF MODIFICATION, EDITING AND REPRODUCTION TO DATA IS STRICTLY PROHIBITED.3. You should use the MATERIALS only by/for yourself. You are NOT ALLOWED TO COPY, DISTRIBUTE, PROVIDE, TRANSPORT THE MATERIALS TO ANY 3RD PARTY OR TO THE PUBLIC including uploading the MATERIALS to internet.4. You should clearly notify the source of the MATERIALS as “NAVER Corp.” when your use the MATERIALS.5. NAVER Corp. DOES NOT GUARANTEE THE ACCURACY, COMPLETENESS, INTEGRITY, QUALITY OR ADEQUACY OF THE MATERIALS, THUS ARE NOT LIABLE OR RESPONSIBLE FOR THE MATERIALS PROVIDED HERE.※ Please be noted that since the MATERIALS should be used within the confines of the voice right owner’s agreement (which was reflected in the LICENSE above), your non-compliance of the LICENSE (for example, using the MATERIAL for commercial use or modifying or distributing the MATERIAL) shall also constitute infringement on the voice right owner’s rights, thus may cause expose you to legal claims from the voice right owner.

3. Model

Seq2seq(  (encoder): EncoderRNN(    (input_dropout): Dropout(p=0.3, inplace=False)    (conv): MaskConv(      (seq_module): Sequential(        (0): Conv2d(1, 32, kernel_size=(41, 11), stride=(2, 2), padding=(20, 5))        (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)        (2): Hardtanh(min_val=0, max_val=20, inplace=True)        (3): Conv2d(32, 32, kernel_size=(21, 11), stride=(2, 1), padding=(10, 5))        (4): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)        (5): Hardtanh(min_val=0, max_val=20, inplace=True)      )    )    (rnn): LSTM(1312, 512, num_layers=3, dropout=0.3, bidirectional=True)  )  (decoder): DecoderRNN(    (input_dropout): Dropout(p=0.3, inplace=False)    (rnn): LSTM(1536, 512, num_layers=2, batch_first=True, dropout=0.3)    (embedding): Embedding(2003, 512)    (attention): Attention(      (conv): Conv1d(1, 512, kernel_size=(3,), stride=(1,), padding=(1,))      (W): Linear(in_features=512, out_features=512, bias=False)      (V): Linear(in_features=1024, out_features=512, bias=False)      (fc): Linear(in_features=512, out_features=1, bias=True)      (tanh): Tanh()      (softmax): Softmax(dim=-1)    )    (fc): Linear(in_features=1536, out_features=2003, bias=True)  ))

The LAS performance (CER(%))

4. Dependency

librosa==0.7.0scipy==1.3.1numpy==1.17.2tqdm==4.36.1torch=1.2.0python-Levenshtein==0.12.0

5. Training and Evaluation

las.pytorch/run_script/data/└──kor_syllable.json└──ClovaCall/    └──raw/        -- 42_0603_748_0_03319_00.wav            ...        -- 42_0610_778_0_03607_01.wav    └──clean/        -- 42_0603_748_0_03319_00.wav           ...        -- 42_0610_778_0_03607_01.wav    └──train_ClovaCall.json    └──test_ClovaCall.json└──Dataset2/    └──sub1/        -- audio1.wav        -- audio2.wav            ...    └──sub2/        -- audio1.wav        -- audio2.wav            ...    └──train_Dataset2.json/    └──test_Dataset2.json/

TRAIN_NAME='train_ClovaCall'TEST_NAME='test_ClovaCall'LABEL_FILE=data/kor_syllable.jsonDATASET_PATH=data/ClovaCall/cleanCUR_MODEL_PATH=models/ClovaCall/LAS_ClovaCallpython3.6 -u las.pytorch/main.py \--batch_size 64 \--num_workers 4 \--num_gpu 1 \--rnn-type LSTM \--lr 3e-4 \--learning-anneal 1.1 \--dropout 0.3 \--teacher_forcing 1.0 \--encoder_layers 3 --encoder_size 512 \--decoder_layers 2 --decoder_size 512 \--train-name $TRAIN_NAME --test-name-list $TEST_NAME \--labels-path $LABEL_FILE \--dataset-path $DATASET_PATH \--cuda --save-folder $CUR_MODEL_PATH --model-path $CUR_MODEL_PATH/final.pth

Run train

cd run_script./run_las_asr_trainer.sh

Run eval

cd run_script./run_las_asr_decode.sh

6. Code license

Copyright (c) 2020-present NAVER Corp.Permission is hereby granted, free of charge, to any person obtaining a copyof this software and associated documentation files (the "Software"), to dealin the Software without restriction, including without limitation the rightsto use, copy, modify, merge, publish, distribute, sublicense, and/or sellcopies of the Software, and to permit persons to whom the Software isfurnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included inall copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THEAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS INTHE SOFTWARE.

7. Reference

8. How to cite

@article{ha2020clovacall,  title={ClovaCall: Korean Goal-Oriented Dialog Speech Corpus for Automatic Speech Recognition of Contact Centers},  author={Jung-Woo Ha, Kihyun Nam, Jingu Kang, Sang-Woo Lee, Sohee Yang, Hyunhoon Jung, Eunmi Kim, Hyeji Kim, Soojin Kim, Hyun Ah Kim, Kyoungtae Doh, Chan Kyu Lee, Nako Sung, Sunghun Kim},  journal={arXiv preprint arXiv:2004.09367},  year = {2020}}