Repository files navigation

This repository provides the code forJapanese NLI model, a fine-tuned masked language model.

The model showed performance comparable with those reported inJGLUE [Kurihara et al. 2022] andJSICK [Yanaka and Mineshima 2022] papers, in terms of overall accuracy:

• Hitomi Yanaka and Koji Mineshima.Compositional Evaluation on Japanese Textual Entailment and Similarity. TACL2022.
• Kentaro Kurihara, Daisuke Kawahara, and Tomohide Shibata.JGLUE: Japanese General Language Understanding Evaluation. LREC2022.
• Nils Reimers and Iryna Gurevych.Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP-IJCNLP2019.
• Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov.Unsupervised Cross-lingual Representation Learning at Scale. ACL2020.

Yes, we tested only a single run :(

torch.manual_seed(0)random.seed(0)np.random.seed(0)

1. JSICK
2. JGLUE

We converted string label into integer using the following mapping:

label2int= {"contradiction":0,"entailment":1,"neutral":2}

We mimickedbatch_size=128 using gradient accumulation32 * 4 = 128.

batch_size=32,shuffle=True,epochs=3,accumulation_steps=4,optimizer_params={'lr':5e-5},warmup_steps=math.ceil(0.1*len(data)),