AI vendor due diligence: questions to ask before signing

An AI vendor’s demo tells you what the product can do under favourable conditions. Due diligence should establish what your business can rely on when the inputs are messy, the team is busy and the result matters.

For a leadership team evaluating AI software, begin with a narrow decision: which workflow should improve, how will improvement be measured, and who remains responsible when the system is wrong? The checklist below is a practical purchasing framework, rather than a certification or a claim that a vendor is compliant.

1. Define the business problem before the product

Write a one-page brief before inviting proposals. Describe the current workflow, the people involved, the available data and the operational consequence of mistakes. Include a baseline that can be measured using your own records.

A useful brief might say: “Reduce the time spent preparing draft maintenance summaries, while keeping technical approval with the maintenance lead.” That is more testable than “make operations AI-powered.” It also gives competing vendors the same problem to solve.

Separate the required outcome from a preferred implementation. You may discover that better data capture or a simpler workflow solves the problem without buying a new platform.

2. Ask where your data goes

Request a data-flow explanation covering the application, model providers, subcontractors, storage locations and support access. Ask which categories of data the product needs and which it can operate without.

• Can customer data be used for model training, and under which settings or contractual terms?
• How are retention, deletion and backup handling documented?
• Which staff or subprocessors can access the data?
• What changes when the product is used in another country or business unit?
• Can you restrict the pilot to synthetic or appropriately de-identified inputs?

These questions apply to buyers across the GCC, EU and US, but the applicable contractual and legal requirements depend on the actual jurisdictions and data. Have your responsible specialists assess those requirements; a generic checklist cannot decide them.

3. Test the work, including the failures

Build a small evaluation set from representative tasks. Include ordinary examples, ambiguous inputs, missing information and cases where the correct behaviour is to ask a person for help. Agree on acceptance criteria before the pilot begins.

Assess the whole workflow. A plausible answer that requires extensive checking may create less value than the demo suggests. Record review time, correction work and escalation behaviour alongside speed.

NIST’s AI Risk Management Framework provides voluntary guidance for considering trustworthiness across AI design, development, use and evaluation. It is a useful reference for structuring the risk conversation; referencing it does not prove that a vendor’s product is safe or certified.

4. Understand operational ownership

Ask who will administer access, monitor usage, approve changes and handle incidents. The vendor may run the software, but your company still needs an owner for the business process.

Clarify what happens when a model changes, a service becomes unavailable or a result reaches the wrong person. Request the support escalation path and the information available for investigating a disputed output. Agree which tasks require human approval and how users can override or stop the system.

If the workflow cannot continue without the tool, test the fallback before expanding adoption.

5. Price the complete workflow

Compare a cost scenario that includes subscription or usage fees, implementation, integration, data preparation, staff training, ongoing review and exit work. Ask vendors to show how the price changes with actual users, tasks and usage patterns.

Avoid treating an advertised unit price as the total cost. For example, a low-cost generation step can still leave the business paying for substantial verification. Use a range of plausible workloads and make the assumptions visible to the decision maker.

6. Preserve an exit path

Before signing, ask how you can export your data, prompts, workflow configuration and relevant records. Distinguish what you own from what is licensed or available only within the vendor’s service.

Set a bounded pilot, a named sponsor and a decision date. The outcome should be one of three choices: proceed against agreed criteria, redesign the pilot around a specific weakness, or stop. An open-ended pilot is a poor substitute for a purchasing decision.

A practical approval pack

Bring five items to the approval meeting: the problem brief, data-flow explanation, evaluation results, full-cost scenarios and an exit plan. Each should have an accountable owner and unresolved questions clearly marked.

If the discussion still centres on how impressive the product looks, return to the workflow. The point of AI procurement is to make a business process work better under real conditions. For capital-intensive purchases, the same discipline extends to comparing equipment lifecycle costs.