Repository files navigation

Use Convolutional Recurrent Neural Network to recognize the Handwritten line text image without pre segmentation into words or characters. Use CTC loss Function to train.More read thisMedium Post

Deep Learning self extracts features with a deep neural networks and classify itself. Compare to traditional Algorithms it performance increase with Amount of Data.

• Project consists of Three steps:
  1. Multi-scale feature Extraction --> Convolutional Neural Network 7 Layers
  2. Sequence Labeling (BLSTM-CTC) --> Recurrent Neural Network (2 layers of LSTM) with CTC
  3. Transcription --> Decoding the output of the RNN (CTC decode)

1. Tensorflow 1.8.0 ; You can upgrade to Tensorflow v2 with thislink
2. Flask
3. Numpy
4. OpenCv 3
5. Spell Checkerautocorrect >=0.3.0pip install autocorrect

• IAM dataset download fromhere
• Only needed the lines images and lines.txt (ASCII).
• Place the downloaded files inside data directory

To Train the model from scratch

$ python main.py --train

To validate the model

$ python main.py --validate

To Prediction

$ python main.py

Run in Web with Flask

$ python upload.pyValidation character error rate of saved model: 8.654728%Python: 3.6.4 Tensorflow: 1.8.0Init with stored values from ../model/snapshot-24Without Correction clothed leaf by leaf with the dioappoistmestWith Correction clothed leaf by leaf with the dioappoistmest

Prediction output on IAM Test Data

Prediction output on Self Test Data

See the projectDevnagari Handwritten Word Recognition with Deep Learning for more insights.

• Using MDLSTM to recognize whole paragraph at onceScan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention
• Line segmentation can be added for full paragraph text recognition. To segment a paragraph into lines, you can use an A* path planning algorithm or CNN model.
• Better Image preprocessing, such as: reducing background noise to handle real-time images more accurately.
• Better Decoding approach to improve accuracy. Some of the CTC Decoder foundhere

If you use any part of this project in your work, please cite:

@techreport{Handwritten-Line-text-recognition-using-deep-learning-2019,  title={Handwritten Line Text Recognition},  author={Gautam Sushant},  institution={Tribhuvan University},  year={2019}}

Feel Free to improve this project with a pull Request.

• This is a work from my undergraduate thesis in computer engineering at Tribhuvan University in 2019.