Digital Transformation
No matter your line of work, you will likely have experienced some form of Document Capture. Essentially, it’s about digitally extracting data and information from documents (paper or digital) and passing it into downstream systems and applications to support ongoing processes.
This technology has been in use for many years. For example, accounts payable departments use document capture to extract data from incoming invoices automatically. HR departments use it to streamline the new hire onboarding process. And government agencies use it to ingest and process thousands of forms every month.
To say that any of these document capture exercises are “intelligent” is stretching things a little. Until now, that is.
Recent technology advances in diverse areas such as machine vision, robotic process automation (RPA), and natural language processing have combined with regular document capture to deliver new benefits to end-users.
The analyst community has many names for this new collective, from Cognitive Capture to Capture 2.0. But the term that we think captures the new space is Intelligent Document Processing (IDP). Analyst firm Everest Group estimates the IDP market to be worth around $700 Million in 2020, with projected growth of between 55% and 65% in 2021. So, whatever you want to call it, it’s a growing market.
Read Now: Why IDP is the onramp to automation for banks and credit unions
But what exactly is intelligent document processing? And how is it different from the clunky, error-prone document capture we have been forced to use in the past? And why is the market snowballing? Let’s explore.
The Three Stages of IDP
IDP is a natural evolution of capture. Rather than being a revolutionary new technology, IDP takes on the core raison d’etre of capture (to extract data and information from documents) and supercharges it by incorporating AI and intelligence to reimagine every aspect of the traditional capture flow.
In a recent Cognitive Capture report, Alan Pelz-Sharpe of Deep Analysis explains that IDP has three steps; Acquire, Understand, and Integrate.
These steps focus entirely on the overall goal of IDP rather than how we achieved the goal. This shift offers vast scope to utilize different tools and technologies at each stage. An IDP platform may incorporate RPA, AI, OCR, ML, Natural Language Processing (NLP), and any other technology for processing content.
The ultimate aim of Intelligent Document Processing is achieving a comprehensive, end-to-end capture experience. This means rapidly and efficiently delivering information from the capture process directly into systems or processes downstream. And it does so with the minimum amount of human intervention possible.
In other words, IDP is a perfect example of intelligent automation in action.
Core Capabilities of IDP
Every intelligent document processing system is unique and can incorporate various tools and features into the digital back-office. However, across the three phases of IDP (acquire, understand, and integrate), every solution needs to deliver a core set of capabilities.
Acquire
The first phase of any capture process is the actual “capture” of the content. Popular document types may include emails, web forms, pdfs, and paper documents such as inbound physical mail or even faxes.
For IDP, acquisition is the collection and pre-processing of content, which is made ready for interrogation in the understanding phase. This step in the IDP process can include:
- Image/document pre-processing
- Image enhancement, including de-skewing and cleansing
- Removal of separator pages, email headers, etc.
- Batching of content
- Creation of blocks of similar types of documents for type-specific processing
Understand
Capture systems have incorporated simple content recognition and data extraction for many years. For example, optical character recognition (OCR) is well-known for turning non-digital documents into machine-readable versions.
These tools cannot understand data. They don’t know if the characters “Tim” are the name of a person, a company name, or the word “Time” misspelled. This limitation forces many OCR users to check the results of each scan and data extraction to confirm that the system got it right.
Today’s “intelligent” capture tools incorporate machine vision, machine learning, and natural language processing tools to do much more. These models proactively “read” the data, classify it, and place it into various business systems where workers can see and act on it to make better, faster business decisions. And because machines never get tired, these new platforms can deliver error-free repeatable results 24 X 7.
In essence, processing the 10,000th document will be equally accurate – or even more accurate given the added benefit of machine learning — than the first document processed through the system.
This new form of intelligent capture (or Intelligent Document Processing) goes a long way in helping organizations become more agile. They can:
- More accurately classify documents into document types such as contracts, invoices, applications, etc.
- Recognize subpages within documents that do not need processing or that may have additional meta data in them, e.g. email headers, separator pages, terms, conditions, etc.
- Improve the accuracy of externally provided data by validating it against third party information. Think of address validation, credit scores, etc.
- Extract critical data from nested tables within documents to read things like invoice line-items, etc.
- Validate images to ensure accuracy, such as validating company logos against supplier details
In addition to these process efficiencies, the best IDP platforms drive content processing at an even more intelligent level.
For example, instead of documents following a linear path through processing, IDP often takes multiple passes at each document with different tools and techniques. As a result, it extracts and classifies more data each time until it has collected all the relevant data. This subtle nuance delivers significantly more output in less time and provides organizations with faster, more actionable insights.
Suppose during our capture process, we recognize a document as a loan application. The data we need to extract from a loan application is different from what we might pull from a contract. Using IDP, we can intelligently route a document through a series of steps to extract data needed for a loan application.
Sure, we can achieve this result using traditional capture systems, but only with a lot of work and manual configuration. Intelligent document processing platforms enable this with minimal configuration. These more “intelligent” capture capabilities allow businesses to collect cleaner and more specific data. The result: reduced downstream errors, minimized risk, and better overall process efficiency.
Integrate
Traditional capture systems often sit in isolation from the rest of the business. Their purpose was to scan documents, extract, and drop the resulting document and extracted data into a document management system.
However, the capabilities of modern IDP platforms are much broader, especially when it comes to integrating into a wide range of downstream conduits.
Modern IDP solutions produce two critical outputs when processing documents:
- Extracted data
- As per traditional capture solutions, the data extracted by the process is saved and passed to downstream applications or workflows. Typical outputs include invoice totals and supplier details for an invoice, or applicant name and type of loan for a loan application.
- Understood metadata
- Using embedded AI tools, IDP can make informed judgments and predictions based on the content and data it sees during extraction. For example, the IDP process can automatically determine the type of the document, identify related documents or data within an external ECM system, or even predict whether a loan should be approved or not. This capability has the potential to drive a significant competitive advantage if used effectively.
To deliver real benefit, the IDP solution needs to pass the extracted and understood metadata downstream to appropriate business tools and processes, such as:
- Enterprise Content Management (ECM) systems
- Core business systems, such as accounts payables systems, ERP, CRM, HRMS, etc.
- Internal databases and custom applications
- Workflows such as loan applications, customer onboarding processes, or claims management systems.
By delivering connectivity to multiple business applications, modern intelligent document processing becomes a deeply integrated part of the business ecosystem, not a separate and isolated exercise.
The Future Is Intelligent
Despite having access to document capture tools for many years, few organizations feel that they have solved the problem of getting information fed efficiently into their business systems and processes. Technologies such as RPA, zonal recognition for OCR, and others are all touted as potential solutions. None have fully delivered.
Intelligent document processing looks set to succeed where all have failed. By harnessing the best of existing technologies and combining them in an easy-to-use, integrated, intelligent platform, IDP is able today to solve a wide range of real back-office business problems once and for all.
The explosion of growth seen by analysts such as Everest Group confirms that IDP will cross the chasm from helpful technology to an essential business toolset. Improved productivity, process efficiency, customer experience, and overall ROI mean that now is the time for a serious look at IDP.
