optical character recognition
In plain wordsSoftware that picks out the letters and words in a picture — a road sign, a handwritten note, a page of an old book — and turns them into real text a computer can copy, search or read aloud.
Machine reading of text that exists only as an image, in a scan, photograph, screenshot or video frame, yielding the corresponding sequence of characters for software to index, edit, translate or speak. A traditional pipeline cleans the image (binarization, deskewing), analyzes the layout to find text blocks and lines, segments and recognizes characters or words, and corrects errors with a dictionary or language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own.. Early commercial systems matched characters against templates for particular fonts; later ones used hand-crafted features and statistical classifiers. Since the mid-2010s, recognizers have typically read whole text lines with a convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. and recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood. trained with CTCconnectionist temporal classification, n.: A training method that lets a network learn to transcribe speech or handwriting without being told exactly where each letter or sound begins and ends., or with a transformer encoder–decoder, after a separate text-detection stage. Scene text recognition handles text in natural photographs (signs, packaging), which is curved, blurred and varied in style. Since 2023, vision-language modelsvision-language model, n.: An AI model that understands images and text together — it can describe a photo, answer questions about a chart or find the picture that matches a sentence. increasingly read text directly as part of document understandingdocument understanding, n.: Using AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed., often with no separate OCR step. OCR quality is measured by character and word error rates; it underpins document digitization, reading aids for blind people, check and mail processing, license-plate reading and the creation of text datasets from scanned books.
The OCR confuses 'rn' with 'm' on the faded invoices, so totals still need human review.
Origin
Reading machines for printed characters were invented in the 1910s–30s (Emanuel Goldberg; Gustav Tauschek's 1929 patent). David Shepard's 1951 'Gismo' led to commercial readers in the 1950s; 'optical' contrasts them with magnetic-ink character recognition (MICR), adopted for bank checks from the late 1950s. Ray Kurzweil's 1976 reading machine for blind users recognized text in many fonts.
Usage note
OCR transcribes text but does not interpret it; extracting meaning, structure and fields is document understandingdocument understanding, n.: Using AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed.. Errors concentrate in degraded scans, rare scripts and handwriting, so accuracy measured on clean benchmarks overstates real-world performance.
Related
- document understandingdocument understanding, n.: Using AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed.
- connectionist temporal classificationconnectionist temporal classification, n.: A training method that lets a network learn to transcribe speech or handwriting without being told exactly where each letter or sound begins and ends.
- vision-language modelvision-language model, n.: An AI model that understands images and text together — it can describe a photo, answer questions about a chart or find the picture that matches a sentence.
- MNISTMNIST, prop. n.: A famous collection of 70,000 small images of handwritten digits used to test image-recognition programs. Getting a model to read MNIST digits is the 'hello world' of machine learning.
- word error rateword error rate, n.: The standard score for speech recognition: the share of words a system gets wrong compared with a correct transcript, counting wrong, missing and extra words. Lower is better.
- pattern recognitionpattern recognition, n.: The ability of a computer, or a brain, to spot regularities in data and sort things into categories, such as recognizing handwritten letters, faces or spoken words.
Sources
- Smith, R. (2007). An Overview of the Tesseract OCR Engine. Proceedings of ICDAR 2007.
- Shi, B., Bai, X. & Yao, C. (2017). An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition. IEEE TPAMI 39(11), 2298–2304.
Mentioned in
CAPTCHACAPTCHA, n.: A small test on websites, like typing wavy letters or clicking every picture with a traffic light, meant to be easy for people but hard for computer programs, so as to keep automated bots out., document understandingdocument understanding, n.: Using AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed., pattern recognitionpattern recognition, n.: The ability of a computer, or a brain, to spot regularities in data and sort things into categories, such as recognizing handwritten letters, faces or spoken words., Ray KurzweilRay Kurzweil, prop. n.: An American inventor of reading machines and speech technology who became the best-known prophet of the 'singularity', predicting human-level AI by 2029., robotic process automationrobotic process automation, n.: Software 'robots' that copy the clicks and typing a person would do in office computer programs — such as moving data between forms — to automate repetitive paperwork., word error rateword error rate, n.: The standard score for speech recognition: the share of words a system gets wrong compared with a correct transcript, counting wrong, missing and extra words. Lower is better., connectionist temporal classificationconnectionist temporal classification, n.: A training method that lets a network learn to transcribe speech or handwriting without being told exactly where each letter or sound begins and ends.