Overview

Data

fromdatasetsimportload_datasetjaquad_data=load_dataset('SkelterLabsInc/JaQuAD')

Baseline

Usage

fromtransformersimportAutoModelForQuestionAnswering,AutoTokenizerquestion='アレクサンダー・グラハム・ベルは、どこで生まれたの?'context='アレクサンダー・グラハム・ベルは、スコットランド生まれの科学者、発明家、工学者である。世界初の>実用的電話の発明で知られている。'model=AutoModelForQuestionAnswering.from_pretrained('SkelterLabsInc/bert-base-japanese-jaquad')tokenizer=AutoTokenizer.from_pretrained('SkelterLabsInc/bert-base-japanese-jaquad')inputs=tokenizer(question,context,add_special_tokens=True,return_tensors="pt")input_ids=inputs["input_ids"].tolist()[0]outputs=model(**inputs)answer_start_scores=outputs.start_logitsanswer_end_scores=outputs.end_logits# Get the most likely start of the answer with the argmax of the score.answer_start=torch.argmax(answer_start_scores)# Get the most likely end of the answer with the argmax of the score.# 1 is added to `answer_end` because the index of the score is inclusive.answer_end=torch.argmax(answer_end_scores)+1answer=tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end]))# answer = 'スコットランド'

Limitations

Reference

@misc{so2022jaquad,title={{JaQuAD: Japanese Question Answering Dataset for Machine Reading Comprehension}},author={ByungHoon So and Kyuhong Byun and Kyungwon Kang and Seongjin Cho},year={2022},eprint={2202.01764},archivePrefix={arXiv},primaryClass={cs.CL}}

LICENSE

Have Questions?