Repository files navigation

The official code of TPS_PP (IJCAI 2023)Paper Link

TPS++, an attention-enhanced TPS transformation that incorporates the attention mechanism to text rectification for the first time. TPS++ builds a more flexible content-aware rectifier, generating a natural text correction that is easier to read by the subsequent recognizer. This code is based on MMOCR 0.4.0 (Documentation ) withPyTorch 1.6+.

• NRTR + TPS_PP
• CRNN + TPS_PP
• ABINet-LV + TPS_PP

Please refer toInstall Guide.

Please seeGetting Started for the basic usage of MMOCR 0.4.0.

The specific configuration of the dataset for training and testing can be found hereDataset Document

testing ├── mixture│   ├── icdar_2013│   ├── icdar_2015│   ├── III5K│   ├── ct80│   ├── svt│   ├── svtptraining├── mixture│   ├── Syn90k│   ├── SynthText

Get the pretrained models fromBaiduNetdisk(passwd:cd9r),GoogleDrive.checkpoint model inmodel/xxx/latest.pth, pre-train model inpre_train/xxx/latest.pth

First, the model needs to be pre-trained using without TPS_PP (pre-train), and then trained end-to-end with a network that incorporates TPS_PP (checkpoint). * denotes the performance of the implemented code. checkpoint model inmodel/xxx/latest.pth, pre-train model inpre_train/xxx/latest.pth.

Please refer to the training configurationTraining Doc

Setp 1 : DownloadNRTRpre_train/nrtr/latest.pth inmmocr_ijcai/nrtr/latest.pth

#Step 2PORT=1234 ./tools/dist_train.sh configs/textrecog/nrtr/nrtr_tps++.py ./ckpt/ijcai_nrtr_tps_pp 4           --seed=123456 --load-from=mmocr_ijcai/nrtr/nrtr_latest.pth

Please refer to the testing configurationTesting Doc

This code is based onMMOCR

If you find our method useful for your reserach, please cite

@article{zheng2023tps++,  title={TPS++: Attention-Enhanced Thin-Plate Splinefor Scene Text Recognition},  author={Zheng, Tianlun and Chen, Zhineng and Bai, Jinfeng and Xie, Hongtao and Jiang, Yu-Gang},  journal={IJCAI},  year={2023}}

This project is released under theApache 2.0 license.