The AI Vendor Security Questionnaire That Actually Binds

Nearly every enterprise buyer now sends an AI vendor a security questionnaire before signing. Few get back an answer that constrains what the vendor can do with their data next quarter. The document that lands in the vendor's inbox is usually a familiar one: a Standardized Information Gathering workbook or a Consensus Assessments Initiative Questionnaire, more than 300 rows long, with a short AI section bolted to the end. The vendor's security team fills in the cells, attaches a SOC 2 report, and the deal moves to legal. A quarter later the vendor swaps the foundation model under the product, and not one of the answers you collected still describes the system you bought.

That is the problem with how enterprise buyers assess AI vendors today. The questionnaire was built for deterministic software, so it produces a point-in-time attestation about a system that refuses to stay at a point in time. We have reviewed vendor assessment packets across financial services, healthcare, and SaaS procurement. The pattern holds across all three: the questionnaire is treated as a gate to clear, not a contract to enforce, and its answers never bind to anything the vendor owes you.

What the questionnaire is actually for

A security questionnaire has one job worth the cost of running it. That job is to surface the facts you will write into the contract. Every question whose answer never becomes a binding term in the data processing agreement or the master services agreement is overhead. It generates work for two security teams and protects no one.

Hold the standard AI vendor packet against that standard and most of it fails. A buyer collects 300 answers, scores them, files the workbook, and signs a contract that references none of them. The vendor attested that customer data is not used for training. Good. Is that promise in the signed agreement, with a defined remedy if it is broken? Usually not. The attestation lives in a spreadsheet nobody will open again until the renewal, and the spreadsheet is not the contract.

Here is the counter-take we offer, and it is the one procurement teams resist most. Stop trying to send a longer questionnaire. The answer to a weak assessment is not 40 more AI questions appended to the SIG. It is a short set of questions whose answers you refuse to accept unless they appear, verbatim in substance, as terms you can enforce. Six binding answers beat 300 scored ones.

Why the generic questionnaire misses AI risk

The SIG and the Consensus Assessments Initiative Questionnaire are good instruments for what they were designed to assess. The Cloud Security Alliance's Cloud Controls Matrix covers access control, encryption, change management, and the rest of the deterministic-software control set with real rigor. None of that work is wasted. It simply does not reach the risks that make an AI vendor different from a conventional SaaS vendor.

Three facts about AI products break the assumptions the questionnaire is built on.

The first is that your data is the product's fuel, not just its payload. A conventional SaaS vendor stores your records and gives them back. An AI vendor may run your prompts and documents through a model that learns, through a provider the vendor does not control, and may retain that content to improve the service. The question is no longer "is my data encrypted at rest," it is "does my data leave the boundary I think it lives in, and does it change a model other customers will use."

The second is that the system is not stable. When the core of a product is a foundation model accessed through an API, the vendor can change it on the provider's schedule or their own. Model providers have retired or deprecated model snapshots within 12 months of release, and buyers almost never learn of the swap. A SOC 2 report describing the environment as of last June says nothing about the model serving your traffic today.

The third is that the privilege boundary moved. Agentic features let the product call tools, query systems, and take actions on your behalf. The security boundary is no longer the data the vendor stores; it is what the vendor's agent can cause to happen inside your tenant. A questionnaire row asking about password rotation does not touch this.

The six questions that carry the weight

We route AI vendor assessment to six questions. Each one is written so the answer has to become a contract term, not a checkbox. If a vendor cannot answer in a way you can enforce, that is the finding.

Does our data train your models, and is the answer in writing? The most common attestation in the market is "we do not train on customer data." Accept it only as a signed term with a remedy. Pin down every path: prompts, uploaded documents, retrieval content, and human review of outputs. A vendor who uses a third-party model provider has to bind that provider too, because the no-training promise is only as good as the weakest subprocessor.

Which models and providers sit under the product, and can you change them silently? Ask for the named models and the named providers, and ask who they fall back to under load. Then ask the question that matters more: what notice do we get before you change them. Treat the model supply chain the way NIST's Cybersecurity Supply Chain Risk Management guidance treats any other critical dependency. A model swap can change accuracy, data residency, and refusal behavior at once.

Where does our data live, how long is it kept, and how is it separated from other tenants? Residency, retention, and isolation are three answers, not one. A vendor who retains prompts for 30 days for abuse monitoring is making a defensible choice; a vendor who cannot tell you the retention window is telling you something else. Get the number, get the deletion path, and get the tenant-isolation model in enough detail to know whether your data and a competitor's data share an index.

What can the system send outbound, and who authorizes an action? For any product with agentic or tool-calling features, the real question is what the agent can do without a human in the loop. Which systems can it reach, what can it write, what can it send. The answer defines your blast radius if the model is manipulated through a crafted input, and it belongs in the contract as a scoped list, not a capability the vendor expands at will.

Can we evaluate and audit the system, not just read a certificate? A certificate attests to a process; an evaluation tests the thing you are buying. Reserve the right to run your own adversarial tests against the model in a non-production tenant, and the right to an audit when a term is in dispute. Vendors who treat a SOC 2 report as the end of the conversation are the ones most worth testing.

What happens during an incident or a model deprecation, and who pays? Two events you can predict: a security incident, and the vendor retiring the model you approved. For incidents, pin down notification timelines and the vendor's cooperation obligations. For deprecations, decide now whether a model change triggers re-approval on your side, and who pays for the re-evaluation. These terms are negotiable at procurement and close to impossible to add later.

Make the answers binding

A questionnaire answer is a claim. A contract term is a claim with a remedy attached. The work that separates a real assessment from theater is the step that moves each of the six answers out of the spreadsheet and into the signed agreement.

For data handling, retention, and isolation, the home is the data processing agreement. For model provenance, change notice, and deprecation, the home is a schedule to the master services agreement that most vendors have never been asked to produce; ask for it anyway. For incident cooperation, the home is the incident and breach clause, with timelines that match your regulatory obligations rather than the vendor's convenience. The EU AI Act brings documentation and transparency obligations for high-risk systems, with the bulk applying from August 2026, and a vendor who cannot support that documentation is a vendor whose compliance debt becomes yours.

An emerging management-system standard, ISO/IEC 42001, gives you a second lever. A vendor certified against it has already built the internal evidence your six questions ask for, which shortens the gap between the questionnaire answer and the contract term.

The scoring trap and other failure modes

The dominant failure mode in AI vendor assessment is the scoring ritual. A team tallies 300 answers into a risk score, the score clears a threshold, and the threshold becomes the decision. The number feels objective. It launders a few genuinely load-bearing answers in with hundreds of irrelevant ones, so a vendor can fail the only question that mattered, the training question, and still clear the bar on volume. We have watched a vendor score green while quietly retaining every prompt for model improvement.

A second failure mode is assessing the vendor and ignoring the vendor's providers. The company you contract with may be a thin layer over a foundation model it does not operate. Your no-training term, your residency term, and your retention term all have to flow down, or they are promises the vendor cannot keep.

A third is one-time assessment. Because the model under the product changes, a questionnaire answered at signing and never revisited describes a system that no longer exists. Re-assessment has to be tied to the event that invalidates the old answer, which is the model change, not the calendar. Map the control plane to the NIST AI Risk Management Framework so that re-assessment is a defined trigger, not an annual ritual that misses the change by eleven months.

Where to start

If your AI vendor assessment today is a SIG with an AI tab, four moves will put it on firmer ground this quarter.

1. Cut the AI assessment down to the six binding questions and write each one so its answer has to land in a contract. Keep the SIG for the deterministic controls it assesses well; stop pretending its AI tab does the AI work.
2. Pull your last three signed AI vendor contracts and check whether the training, retention, and model-change answers actually appear as terms. Where they do not, you have an exposure, not an assessment.
3. Add a model-change trigger to every AI contract, with a defined notice period and a decision on whether the change forces re-approval. Decide who pays for the re-evaluation before you need the answer.
4. Reserve the right to run your own evaluation against the model in a non-production tenant, and use it on your two highest-stakes vendors before the next renewal. A certificate is a starting point, not the finish line.

The buyers who get this right have stopped grading AI vendors and started contracting with them. The questionnaire stops being a filing cabinet of attestations and becomes the first draft of the terms you will enforce when the model changes, the data moves, or the agent does something nobody approved. Build it that way, and procurement is doing security work. Skip it, and you are collecting answers that bind no one.