What is Advanced Data Extraction (ADE)?

OCR and Data Extraction

A big part of any OCR solution is the process of data capture and extraction.  Most document capture applications provide the ability to process the converted text and provide the extraction of expressions from the text.  So how can this  help?  Well, the ability to parse the OCR text provides automation, and allows you to populate fields based on what you find.

An example might be a form that has DOB: 1/2/1968

You want to extract everything to the right of DOB: from a document.  You can do this with an ADE engine.

OCR and the Right Settings

What DPI should be set for optimal OCR Accuracy?

So. I get this question all the time and decided it might be good to post about it.  What is the best DPI setting for Optical Character Recognition (OCR)?

I have been at clients that erroneously believe the higher the DPI, the beeter the results, and feel pain whenever I see an OCR Scanner set beyond 300 DPI, and some even at 600 DPI!!  Holy cow, how do you handle those file sizes?

The fact remains that almost all OCR engines on the market are tuned and optimized for 300DPI for optimal conversion and recognition.  Going beyond this will provide no better results, and significantly increase your file size exponentially.  Most Document Capture companies provide image processing prioer to OCR that will allow you to scan at 200 DPI, with fairly consistent results.

OCR Software Post

OCR Software for Business


8 Things About OCR

OCR Software and Character Correction

Optical Character Recognition and Character Correction

So what is character correction when associated with OCR?  The OCR process provides the recognition and conversion of images to text, and in this process, there can be many characters that can be misidentified throughout the conversion process.  Typically, document capture applications provide the ability to identify commonly misinterpreted characters through a table of correction mappings.  So lets say a particular zone OCR field was designated as numbers only, and the engine interpreted an "l" for a "1" (that is an l for a one).  The correction piece of the recognition engine can provide logic to the OCR process, and make sure the text is properly interpreted. This can be really important, especially in SharePoint OCR environments where you need searchable PDFs in SharePoint.

This is just one of many ways to improve accuracy, but note you will need the right kind of OCR application that allows this feature to be enabled.

Are index fileds really necessary when you have the full text OCR?

Full text OCR

Ah, the old debate, do I just perform optical character recognition on all my scanned documents, make them searchable OCR PDFs, and rely on the OCR to retrieve documents?  Why use index fields when I already have all the converted text?

Index fields, or performing the indexing process, provides structured data about the documents.  This data can be utilized, especially when using document capture software, to link into columns and index fields in your document management system.  Index fields provide faster retrieval, especially if you want to be able to retrieve through specifying several criteria.  Relying on OCR, or the recognized text can get you in trouble.  First of all, you are assuming that the document will alwyas have recognized text, and that all the items that you are searching for are in the text.  Secondly, depdning on the type of OCR format you have, you may have to just find the document, and then open and parse what you are looking for.  This can also lead to false positives in retrieval if many documents have the same terms in their OCR text.

Zone OCR and Accuracy within Recognition Zones

Zone OCR Accuracy

So when doing zone OCR , or Optical Character Recognition on a portion of a page, what features do I need to ensure I have the best possible accuracy.  List below:

• Utilize a document capture application that provides some type of page registration.  The problem with using zone OCR is that most engines utilize a set template of coordinates on the page, and just repeat this "zone" on each page.  If the scanner is off, or the page skewed, you can have erroneous readings.  Page registration gives the recognition engine the ability to anchor a page feature, always referencing the zone from the set coordinates of the feature.
• Utilize a scanning application that provides the ability to perform image processing on the zone prior to running Optical Character Recognition . Removing lines, deshading, despeckling can provide a cleaner zone, and thus improve overall accuracy.
• Some advanced capture applications provide the ability to filter zones based on character sets.  This allows you to interpret the characters within a zone as say, all numbers, or perhaps a date, which provides the engine a more narrower character set for the whole recognition process.  iCapture for example, not only allows character set mapping to zone ocr templates, but also provides auto-correction for the most commonly misinterpreted characters.
• Finally, and highly recommended for the highest level of accuracy, is the ability to set a character matching filter for a zone.  This technology, sometimes called ADE, provides the ability to utilize regular expressions to ensure a match, and lets you over draw the recognition area / zone and filter to your liking.

Why use OCR Software to perform full text conversion of images?

OCR Software

When we scan documents, they are just images, pictures of our paper.  For many organizations, this scanned image is exactly what they need, and a little index information about the document is sufficient to provide them with retrieval capability.

So why take the time and spend the money to utilize OCR Software to convert the scanned document to a searchable format?  Below are some reasons to always perform full text OCR of scanned documents:

1. Always provide every means possible for retrieval.  Just using index fields to search for scanned documents may seem like a fantastic idea, but what if the document is misidentified?  Or the indexer enters incorrect information?  Performing a full text OCR of the document can provide an insurance policy that a document can always be found through full text search.
2. Document Capture software today provides fast reliable OCR.  Most capture software on the market provides the ability to automatically convert the documents to searchable format for a small expense.  Some of the engines on the market can do the conversion at 100+ pages per minute, so there is really not much time wasted in the OCR conversion / recognition process.
3. OCR to PDF for a format that contains both image and text in one container.  Adobe provides the PDF image with hidden text option to give you a seachable file format that contains a pristine image.
4. Plan for the worst case.  Audits...legal issues...sometimes you need to search beyond the index fields, and full text can give you the ability to find the needle in the haystack.

OCR applications give you the means and capabilities to convert images to searhcable formats and there are many reasons to do the full text conversion.