Setup

$ git clone https://github.com/lilianweng/transformer-tensorflow.git$cd transformer-tensorflow$ pip install -r requirements.txt

Train a Model

# Check the help message:$ python train.py --helpUsage: train.py [OPTIONS]Options:  --seq-len INTEGER               Input sequence length.  [default: 20]  --d-model INTEGER               d_model  [default: 512]  --d-ff INTEGER                  d_ff  [default: 2048]  --n-head INTEGER                n_head  [default: 8]  --batch-size INTEGER            Batch size  [default: 128]  --max-steps INTEGER             Max train steps.  [default: 300000]  --dataset [iwslt15|wmt14|wmt15]                                  Which translation dataset to use.  [default:                                  iwslt15]  --help                          Show this message and exit.# Train a model on dataset WMT14:$ python train.py --dataset wmt14

Evaluate a Trained Model

$ python eval.py transformer-wmt14-seq20-d512-head8-1541573730

Implementation Notes