Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (EMNLP 2020)

NotificationsYou must be signed in to change notification settings

INK-USC/MHGRN

Repository files navigation

License: MIT

This is the repo of our EMNLP'20paper:

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question AnsweringYanlin Feng*, Xinyue Chen*, Bill Yuchen Lin, Peifeng Wang, Jun Yan and Xiang Ren.EMNLP 2020.*=equal contritbution

This repository also implements other graph encoding models for question answering (including vanilla LM finetuning).

  • RelationNet
  • R-GCN
  • KagNet
  • GConAttn
  • KVMem
  • MHGRN (or. MultiGRN)

Each model supports the following text encoders:

  • LSTM
  • GPT
  • BERT
  • XLNet
  • RoBERTa

Resources

We provide preprocessed ConceptNet and pretrained entity embeddings for your own usage. These resources are independent of the source code.

Note that the following reousrces can be downloadhere.

ConceptNet (5.6.0)

DescriptionDownloadsNotes
Entity Vocabentity-vocabone entity per line, space replaced by '_'
Relation Vocabrelation-vocabone relation per line, merged
ConceptNet (CSV format)conceptnet-5.6.0-csvEnglish tuples extracted from the full conceptnet with merged relations
ConceptNet (NetworkX format)conceptnet-5.6.0-networkxNetworkX pickled format, pruned by filtering out stop words

Entity Embeddings (Node Features)

Entity embeddings are packed into a matrix of shape (#ent, dim) and stored in numpy format. Usenp.load to read the file. You may need to download the vocabulary files first.

Embedding ModelDimensionalityDescriptionDownloads
TransE100Obtained using OpenKE with optim=sgd, lr=1e-3, epoch=1000entitiesrelations
NumberBatch300https://github.com/commonsense/conceptnet-numberbatchentities
BERT-based1024Provided by Zhengweientities

Dependencies

Run the following commands to create a conda environment (assume CUDA10):

conda create -n krqa python=3.6 numpy matplotlib ipythonsource activate krqaconda install pytorch=1.1.0 torchvision cudatoolkit=10.0 -c pytorchpip install dgl-cu100==0.3.1pip install transformers==2.0.0 tqdm networkx==2.3 nltk spacy==2.1.6python -m spacy download en

Usage

1. Download Data

First, you need to download all the necessary data in order to train the model:

git clone https://github.com/INK-USC/MHGRN.gitcd MHGRNbash scripts/download.sh

The script will:

2. Preprocess

To preprocess the data, run:

python preprocess.py

By default, all available CPU cores will be used for multi-processing in order to speed up the process. Alternatively, you can use "-p" to specify the number of processes to use:

python preprocess.py -p 20

The script will:

  • Convert the original datasets into .jsonl files (stored indata/csqa/statement/)
  • Extract English relations from ConceptNet, merge the original 42 relation types into 17 types
  • Identify all mentioned concepts in the questions and answers
  • Extract subgraphs for each q-a pair

The preprocessing procedure takes approximately 3 hours on a 40-core CPU server. Most intermediate files are in .jsonl or .pk format and stored in various folders. The resulting file structure will look like:

.├── README.md└── data/    ├── cpnet/                 (prerocessed ConceptNet)    ├── glove/                 (pretrained GloVe embeddings)    ├── transe/                (pretrained TransE embeddings)    └── csqa/        ├── train_rand_split.jsonl        ├── dev_rand_split.jsonl        ├── test_rand_split_no_answers.jsonl        ├── statement/             (converted statements)        ├── grounded/              (grounded entities)        ├── paths/                 (unpruned/pruned paths)        ├── graphs/                (extracted subgraphs)        ├── ...

3. Hyperparameter Search (optional)

To search the parameters for RoBERTa-Large on CommonsenseQA:

bash scripts/param_search_lm.sh csqa roberta-large

To search the parameters for BERT+RelationNet on CommonsenseQA:

bash scripts/param_search_rn.sh csqa bert-large-uncased

4. Training

Each graph encoding model is implemented in a single script:

Graph EncoderScriptDescription
Nonelm.pyw/o knowledge graph
Relation Networkrn.py
R-GCNrgcn.pyUse--gnn_layer_num and--num_basis to specify #layer and #basis
KagNetkagnet.pyAdapted fromhttps://github.com/INK-USC/KagNet, still tuning
Gcon-Attngconattn.py
KV-Memorykvmem.py
MHGRNgrn.py

Some important command line arguments are listed as follows (runpython {lm,rn,rgcn,...}.py -h for a complete list):

ArgValuesDescriptionNotes
--mode{train, eval, ...}Training or Evaluationdefault=train
-enc, --encoder{lstm, openai-gpt, bert-large-unased, roberta-large, ....}Text EncoerModel names (except for lstm) are the ones used byhuggingface-transformers, default=bert-large-uncased
--optim{adam, adamw, radam}Optimizerdefault=radam
-ds, --dataset{csqa, obqa}Datasetdefault=csqa
-ih, --inhouse{0, 1}Run In-house Splitdefault=1, only applicable to CSQA
--ent_emb{transe, numberbatch, tzw}Entity Embeddingsdefault=tzw (BERT-based node features)
-sl, --max_seq_len{32, 64, 128, 256}Maximum Sequence LengthUse 128 or 256 for datasets that contain long sentences! default=64
-elr, --encoder_lr{1e-5, 2e-5, 3e-5, 6e-5, 1e-4}Text Encoder LRdataset specific and text encoder specific, default values inutils/parser_utils.py
-dlr, --decoder_lr{1e-4, 3e-4, 1e-3, 3e-3}Graph Encoder LRdataset specific and model specific, default values in{model}.py
--lr_schedule{fixed, warmup_linear, warmup_constant}Learning Rate Scheduledefault=fixed
-me, --max_epochs_before_stop{2, 4, 6}Early Stopping Patiencedefault=2
--unfreeze_epoch{0, 3}Freeze Text Encoder for N epochsmodel specific
-bs, --batch_size{16, 32, 64}Batch Sizedefault=32
--save_dirstrCheckpoint Directorymodel specific
--seed{0, 1, 2, 3}Random Seeddefault=0

For example, run the following command to train a RoBERTa-Large model on CommonsenseQA:

python lm.py --encoder roberta-large --dataset csqa

To train a RelationNet with BERT-Large-Uncased as the encoder:

python rn.py --encoder bert-large-uncased

Toreproduce the reported results of MultiGRN on CommonsenseQA official set:

bash scripts/run_grn_csqa.sh

5. Evaluation

To evaluate a trained model (you need to specify--save_dir if the checkpoint is not stored in the default directory):

python {lm,rn,rgcn,...}.py --modeeval [ --save_dir path/to/directory/ ]

Use Your Own Dataset

  • Convert your dataset to{train,dev,test}.statement.jsonl in .jsonl format (seedata/csqa/statement/train.statement.jsonl)
  • Create a directory indata/{yourdataset}/ to store the .jsonl files
  • Modifypreprocess.py and perform subgraph extraction for your data
  • Modifyutils/parser_utils.py to support your own dataset
  • Tuneencoder_lr,decoder_lr and other important hyperparameters, modifyutils/parser_utils.py and{model}.py to record the tuned hyperparameters

About

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (EMNLP 2020)

Resources

Stars

Watchers

Forks

Packages

No packages published

[8]ページ先頭

©2009-2025 Movatter.jp