Who to trust: How to choose a Document AI provider that’s right for your business?
With so many Document AI, data extraction and contract analytics platforms available – and new ones emerging all the time – the landscape can be confusing. Many companies looking for a better way to access and analyze large amounts of data from documents are unsure how to choose a solution.
In this blog, we guide you through the questions to ask potential vendors, giving you some tips on where to steer the conversation to help you determine which solution best meets your needs.
Vendor Question 1: What can you tell me about the data accuracy that you deliver?
Understandably, this is usually one of the first questions we get asked by prospective clients. After all, improving speed and accuracy are the main reasons, along with cost reduction, that companies turn to us for their data automation needs. First, be wary of anyone claiming 100% data accuracy out of the box. Unless your documents are completely standardized, and you only need to extract quantitative data from them, 100% accuracy from the outset is unrealistic. And if your data is sufficiently structured and organized to allow for instant precision, you probably don’t need a Document AI solution in the first place.
To optimize for accuracy, a machine learning model needs to be trained based on your specific use case, documents and data requirements. So, the answer to your data accuracy question will differ by use case and be dependent on factors such as variation in documents and the complexity and granularity of data required. The provider should be able to share with you the accuracy levels they’ve achieved for clients with the same or similar needs as yours. The accuracy rate should come with some caveats on the volume of training documents and the length of time typically required to achieve that level of performance.
At Eigen, we deliver market-leading accuracy rates typically of more than 80-90% because we apply the best elements of the three approaches to machine learning to drive the best outcomes. In the diagram below, you can see our three-pronged approach to machine learning. Our platform also guides users to review low confidence answers providing a feedback loop that ensures the model is continuously learning on the job.
Vendor Question 2: How can your Document AI solution meet my exact needs?
When sourcing any new technology, your starting point is identifying the problem that you need to solve as well as the people, processes and systems that are impacted by those changes. A provider can’t address your needs if they don’t understand them so share with them as much information as possible about your specific use case (with an NDA in place if required). But no matter how thorough you are in gathering and prioritizing your Document AI requirements, one thing is inevitable; they will change.
- Ask any potential provider what their process is for handling change requests.
- Ask them how often they launch new platform features and functionality and how these are prioritized and rolled out to clients.
- Ask them how they work with internal teams, third parties and integration partners.
- Ask them how easy it is for clients to scale their platform usage up or down.
If you’re in the market for a Document AI platform, you probably want to create efficiencies, drive down costs, automate and accelerate processes or analyze previously inaccessible data. To achieve these goals, you need a solution that can be finely tuned to your exact needs but is also highly adaptable and able to cope with evolving change. If you need the platform to work with other systems and repositories, make sure you choose a provider who has a proven track record in integrations and can offer APIs, plugins or other options.
Eigen has successfully delivered customized solutions and developed integrations for clients across a diverse set of use cases ranging from one-off regulatory exercises to full-scale digital transformation projects. Eigen’s models are built to order to ensure they meet the exact needs of each client. This bespoke approach makes it easier for the client to manage and tweak the model as they are familiar with its set-up and its data output.
Vendor Question 3: How do you ensure my documents and data are secure within your environment?
Another common concern for clients is the security of their data. And rightly so given the sensitivity of their information, their customers’ expectations and the risks and regulations surrounding data handling. If you’re entrusting a third-party with your documents and data, you need to ensure they are taking that responsibility as seriously as you do.
When you ask the question ‘how do you ensure my documents and data are secure within your environment?’ the response should cover a broad spectrum of measures. These should include platform access and security, procedures for data transmission and storage plus ongoing monitoring, reporting, testing and staff training. Ask them if they hold industry-standard certifications and undergo independent security audits. And don’t be afraid to ask the vendor to provide details of their information security credentials (e.g., ISO certifications) and related policy documents (e.g., data protection, disaster recovery and business disruption) to substantiate their claims.
Don’t let the vendor blind you with science or baffle you with jargon. Information security is as much about processes and people as it is about authentication, encryption and data replication. You want to work with a provider who is clear, and upfront about their solution and their working practises. If the vendor is offering a cloud-hosted solution, find out who manages that service and which party is responsible for security.
Eigen holds three ISO certifications to a gold standard, and we are subject to frequent independent audits to ensure we meet the necessary standards of information security management and business security possible to retain these certifications.
Vendor Question 4: What support and training will you provide?
Implementing any new technology or system takes time, effort and money and can be a daunting prospect. In the case of AI, it’s also disruptive by design. You’re not just replacing a legacy system you’re automating processes and changing workflows that have been developed over time. Don’t underestimate the effort involved in implementing Document AI successfully in ways that maximize your ROI. That said the platform itself should be intuitive and straightforward to use once you’ve received training.
The exact services and support you require from a vendor will be dependent on the scope and scale of the implementation project. The complexity of integrations with other systems and your hosting requirements will also have a bearing on the workload. The vendor should provide you with a delivery plan alongside their proposal that sets out the requirements and responsibilities on both sides with an indicative timeline to production. Be sure to clarify what’s required of your internal teams and the vendor to achieve the results and accuracy rate you’ve previously discussed.
However intuitive and easy to use the platform appears you should be offered training, so your team understands how to develop the model for your specific use case. Beware of vendors who provide a self-service solution without any support - you won’t achieve the best outcomes without training. Ideally, you should be provided with a dedicated account manager, around-the-clock technical support and training materials.
Eigen offers its clients a range of additional technical and professional services including APIs, plugins and model development to ensure they get the most out of their Document AI platform while only paying for what they need. All Eigen’s clients have a dedicated Customer Success Manager, receive regular platform upgrades and ongoing maintenance and support.
Vendor Question 5: How long will it take to get up and running with the platform?
There’s no simple answer to this question as the timeframe will vary depending on several factors including the variation in documents, the number of data points and answers required, the complexity of integrations with downstream and upstream systems, the desired data accuracy etc. But this is not a trick question. A provider should be able to give you a relatively accurate estimation based on their knowledge of your use case and experience of implementing similar-sized projects for other clients.
The pre-production implementation phase can run from a few weeks to a few months but should be phased, and project managed, so you know where you are in the rollout process at any given time. Many providers, including Eigen, have developed warmed-up models for specific use cases to give clients a head start. These models are built based on the typical document structure and data requirements for a use case and can significantly reduce the time needed to train a model from scratch. However, the warmed-up model will need additional training to deliver a client’s specific requirements and should not be considered a substitute for developing their own model.
Eigen’s Document AI platform can be trained using very few documents. It can take as few as 2 to 100 documents to successfully train the platform while our competitors often require hundreds or thousands. We’ve developed cutting-edge machine learning techniques that help us to reduce the time to value for our clients.
The Document AI future is now
When you’re evaluating potential vendors asking the right questions is only part of the process. Always request a custom demo using samples of your own documents or those of a similar length and format. Be as specific as you can be about the data points, sections and answers you want extracted from the documents so the demo can include these and give you a better sense of the work required to get you the results you need.
In our next blog, we’ll look at how integrating Document AI can provide your organization with maximum efficiency, resulting in significant benefits to your bottom line.
Want to find out if Eigen is the right Document AI solution for your business?
Request a demo of our platform to find out if we can help you solve your document and data-related challenges.
- World Economic forum 2020
- Gartner Cool Vendor 2020
- AI 100 2021
- Lazard T100
- FT Intelligent Business 2019
- FT Intelligent Business 2020
- CogX Awards 2019
- CogX Awards 2021
- CogX Awards Best AI Product in Insurance
- Forbes The AI 50 2022
- Ai BreakThrough Award 2022