An attention based method for offline handwritten Urdu text recognition

Tayaba Anjum, Nazar Khan
17th International Conference on Frontiers in Handwriting Recognition (ICFHR 2020)

Abstract

Compared to derivatives from Latin script, recognition of derivatives from Arabic hand-written script is a complex task due to the presence of two-dimensional structure, context-dependent shape of characters, high number of ligatures, overlap of characters, and placement of diacritics. While significant attempts exist for Latin and Arabic scripts, very few attempts have been made for offline, handwritten, Urdu script. In this paper, we introduce a large, annotated dataset of handwritten Urdu sentences. We also present a methodology for the recognition of offline handwritten Urdu text lines. A deep learning based encoder/decoder framework with attention mechanism is used to handle two-dimensional text structure. While existing approaches report only character level accuracy, the proposed model improves on BLSTM-based state-of-the-art by a factor of 2 in terms of character level accuracy and by a factor of 37 in terms of word level accuracy. Incorporation of attention before a recurrent decoding framework helps the model in looking at appropriate locations before classifying the next character and therefore results in a higher word level accuracy.

Presentation Video

Files

Paper	Presentation	Code	Dataset

Bibtex

@inproceedings{anjum2020urdu_ohtr,
    author = {Anjum, Tayaba and Khan, Nazar},
    title = {{An attention based method for offline handwritten Urdu text recognition}},
    booktitle = {International Conference on Frontiers in Handwriting Recognition (ICFHR)},
    month = {September},
    year = {2020},
}
		

Acknowledgements

This work was supported by the Higher Education Commission (Pakistan) under Grant 8329/Punjab/NRPU/R&D/HEC/2017. We thank all the undergraduate students at University of Management and Technology (UMT) for agreeing to become scribes for our dataset. We also thank Bilal Rasheed and Faizan Saleem for the tedious process of ground-truth annotations and corrections.