AI Vendor Due-Diligence Question Set

Contents

# AI vendor due-diligence question set

Each question states the evidence tier a satisfactory answer reaches.

## D1 Documentation
1. Supply the model or system card. (T2)
2. Name three uses this system must not be put to. (T2)
3. What are the three most common ways it is wrong, and how often? (T2/T3)
   Follow-up if the answer is qualitative: "over what population was that observed?"

## D2 Evaluation
4. For your headline accuracy claim: what is the denominator, the date, and the
   definition of a correct outcome? (T2)
5. Could a third party reproduce that result from what you publish? (T3)
6. Has anyone outside your company evaluated this system and published? (T4)

## D3 Safety
7. Who red-teamed it, when, and what did they find? (T2/T3)
8. Is human review enforced in code or requested in a prompt? Show me. (T3)

## D4 Fairness
9. What disparity testing has been done, across which cohorts? (T2)
10. Which variables are proxies for protected classes, and how do you know? (T3)

## D5 Privacy
11. Is training on our inputs excluded contractually, not just by policy? (T2)
12. Name every sub-processor that touches our data. (T2)

## D6 Accountability
13. Given a decision made nine months ago, can you tell me which version made it? (T3)
14. What is your deprecation notice period, in writing? (T2)

## D7 Mappability
15. Supply your crosswalk at article level, not framework level. (T2)

(Remaining 25 questions in the full set.)