What does a vendor's SOC 2 report leave unanswered about access to AI answers?

A vendor's SOC 2 report confirms baseline security controls but leaves three AI questions open: it doesn't show whether permissions hold at retrieval time, how model providers use your data, or whether answers are grounded in real sources.

Those gaps are the core SOC 2 limitations for AI. The audit assumes deterministic software and doesn't assess whether a model's outputs are accurate, safe, or appropriate.

This guide explains what the audit tests, where AI sits outside its scope, and which checks to add to your vendor assessment for AI.

What SOC 2 reports actually cover — and where AI falls outside the scope

SOC 2 is an attestation against the AICPA Trust Services Criteria: security, availability, processing integrity, confidentiality, and privacy. None of the five criteria is specific to AI.

Every report is bounded by a scoped system and a covered period. As Code Your Compliance points out, a clean report proves a tested control environment for that scope and window. It doesn't prove the AI product, model path, retention path, or contract boundary you're buying matches what was audited.

Treat vendor privacy statements as claims to verify against the report and the contract. For more on why AI privacy claims aren't the same as audited controls, see this breakdown.

How AI access controls differ from traditional application permissions

Traditional application permissions are binary. A user can open a file in a document tool or can't, and the app checks that right each time.

An AI assistant works differently. One answer can combine a CRM record, a wiki page, and a support ticket, each governed by a different system's access rules. AI access controls have to check every one of those sources against the asking user's rights at retrieval time. IBM found that 97% of organizations reporting an AI-related breach lacked proper AI access controls.

The OWASP Top 10 for LLM Applications 2025 labels this risk LLM08, Vector and Embedding Weaknesses. Tenable's analysis of the 2025 update explains that content retrieved through retrieval-augmented generation (RAG) can reach unauthorized users. Tenable also describes cross-context leakage, which happens when contexts inside a single tenant get confused.

A SOC 2 report won't tell you whether permissions are checked at retrieval time. Permission-aware retrieval, which filters results by each user's source-level rights before the model sees them, needs a separate evaluation.

What to ask vendors about permission enforcement

Three questions expose how a vendor handles permissions inside the AI pipeline:

1. Do you enforce permissions before content enters the model's context, or filter answers after generation?
2. Do permission changes sync in real time or on a schedule, and what's the longest possible lag?
3. Can admins see which sources contributed to a specific answer?

The first question matters most. Superwise's guidance on the OWASP Top 10 for LLM Applications 2025 recommends checking authorization before retrieval and sending the model only fields a task needs.

Where SOC 2 goes silent on data exposure during AI retrieval and generation

AI data security depends on what happens inside the retrieval-augmented generation (RAG) pipeline. A SOC 2 report won't show whether that data path was covered. Each stage copies or reshapes your content in a way a security review should trace:

• Indexing stores copies of source content, so ask where they live and for how long.
• Embedding turns text into vectors, so ask whether vector stores are isolated per customer.
• Semantic search reaches across connected systems, so ask which repositories a query can touch.
• Context assembly selects passages for the model, so ask how much content gets passed along.
• Prompt construction packages everything for the model, so ask what leaves your environment.

The trust boundary often extends past the vendor, and third-party SaaS was the most common source of AI security incidents in IBM's 2025 research. As this DEV Community guide notes, the LLM provider becomes a subprocessor in your data flow. Under the carve-out method, the report names that provider but doesn't test its controls, so you evaluate it separately.

A SOC 2 report also won't tell you whether the vendor trains on your data. AuditBadger advises checking the data processing agreement (DPA) instead. Review that agreement for zero data retention terms and third-party LLM logging policies too.

Exposure at this layer is a recognized concern for data privacy in AI. Sensitive Information Disclosure is LLM02 in the OWASP Top 10 for LLM Applications 2025.

What SOC 2 does not reveal about AI answer accuracy and grounding

SOC 2's processing integrity criterion was written with deterministic software in mind, where the same input yields the same output. Large language models generate probabilistic text. According to Drel, SOC 2 doesn't assess whether a model produces accurate outputs, so a vendor can pass the audit and still return hallucinated answers.

The OWASP Top 10 for LLM Applications 2025 files hallucinations under LLM09, Misinformation, which replaced Overreliance. Grounding, citation quality, and confidence signals need their own review.

The cost shows up in everyday workflows. A support agent quotes a retired refund policy to a customer. A sales rep sends last quarter's price sheet, or an operations lead acts on a stale runbook.

Compliance teams carry the sharpest risk. An analyst who cites a superseded control in an audit response creates a finding the AI never flagged. Ask vendors to show source citations on every answer and to explain what happens when no grounded answer exists.

Why vendor risk management for AI requires frameworks beyond SOC 2

SOC 2 is one input to vendor risk management for AI. The report shows a tested control environment for a scoped system, while AI governance frameworks address how models behave and who's accountable for them. That second layer is often missing: 68% of breached organizations lacked AI governance in IBM's 2026 report.

The NIST AI Risk Management Framework (AI RMF 1.0, released January 2023) covers governance, explainability, and bias. Its four functions translate into vendor questions:

• Govern asks who owns AI risk policy and decisions at the vendor.
• Map asks which use cases, data sources, and users the system touches.
• Measure asks how the vendor tests for accuracy, bias, and explainability.
• Manage asks how identified risks get prioritized, fixed, and monitored.

Two other references extend that coverage. ISO/IEC 42001:2023 is the first international standard for AI management systems. The EU AI Act entered into force on August 1, 2024, with phased obligations that include transparency and human oversight for high-risk systems.

Layered due diligence stacks these together: the SOC 2 report, framework mapping, contract review, and hands-on testing. For further reading on SOC 2 compliance for AI, see this practitioner discussion.

How to evaluate an AI vendor's security posture beyond the audit report

Evaluating an AI vendor's security posture means testing claims against evidence you collect yourself. We recommend five checks to cover what the audit report leaves out:

1. Request a data flow diagram that traces one query from prompt to model provider and back.
2. Revoke a pilot user's access to a document, then ask a question only that document answers.
3. Read the LLM provider contract clauses directly, and ask for notice before subprocessors change.
4. Ask how long AI activity logs are retained and whether they can export to your SIEM.
5. Ask for AI red-team results covering LLM01 Prompt Injection and jailbreak attempts.

The second check tells you where permission enforcement happens. Upstream enforcement keeps the restricted document out of the model's context entirely. Downstream filtering means the model already processed it, and only the visible answer was trimmed.

The fifth check closes a common testing gap. Workstreet points out that traditional penetration tests don't automatically cover prompt injection, jailbreaks, or model behavior evaluations.

Report type matters too. As this DEV Community guide explains, a Type I report assesses control design at a single point in time. A Type II report tests operating effectiveness over a period, so ask for it when making AI tool compliance decisions.

For peer discussion on running these checks, you can browse the Glean Community forum.

Frequently asked questions

What specific aspects of AI access are not covered by SOC 2 reports?

SOC 2 doesn't cover RAG risks such as cross-context leakage within a tenant, whether the vendor trains on your data, or output accuracy. Risks in the OWASP Top 10 for LLM Applications 2025 such as System Prompt Leakage (LLM07) and Improper Output Handling (LLM05) need separate testing too.

How can organizations assess AI-related risks beyond SOC 2?

Pair document review with live testing. Send the vendor an AI security questionnaire built on the OWASP Top 10 for LLM Applications 2025. Then validate the answers in a pilot that uses your own restricted content.

What are the implications of using AI tools without adequate SOC 2 coverage for AI-specific risks?

Without AI-specific checks, restricted data can reach the wrong employees, and prompts can flow to model providers you haven't reviewed. IBM found high levels of shadow AI added USD 670,000 to the average breach cost. When AI agents act in connected systems, LLM06 Excessive Agency in the OWASP Top 10 for LLM Applications 2025 adds the chance of unauthorized changes.

How does SOC 2 address data privacy in the context of AI?

Privacy is one of five SOC 2 criteria, but security is the only required one, so check whether privacy was in scope at all. Even when privacy is in scope, the report won't tell you whether the vendor trains on your data. Ask separately how personal data in prompts and embeddings gets deleted.

You'll learn the most about an AI vendor by testing permissions, contracts, and answer grounding yourself. We enforce permissions upstream of the LLM, so answers use only content each user can access, with citations you can check. Request a demo and we'll walk your security team through how we handle retrieval, retention, and audit logging.