--- title: The anatomy of an LLM response…what compliance professionals need to know is happening under the hood description: Discover the hidden complexities behind LLM responses and why compliance professionals need to approach AI outputs with caution before making decisions. image: https://www.deep-dive.com/hubfs/LLM%20and%20human%20working%20together%20shutterstock_2742687765%20(1).jpg --- [Skip to content](https://www.deep-dive.com/resources/the-anatomy-of-an-llm-response-what-compliance-professionals-need-to-know-is-happening-under-the-hood#main-content) [![DeepDive logo](https://www.deep-dive.com/hubfs/DD_Horizontal%20blue%20on%20white%20SVG.svg)](https://www.deep-dive.com/) - [EDD](https://www.deep-dive.com/enhanced-due-diligence-reports) - [Investigation](https://www.deep-dive.com/investigation-automation) - [How It Works](https://www.deep-dive.com/how-deepdive-works) - [Resources](https://www.deep-dive.com/resources) - [About](https://www.deep-dive.com/about) Open main navigation Close main navigation - [EDD](https://www.deep-dive.com/enhanced-due-diligence-reports) - [Investigation](https://www.deep-dive.com/investigation-automation) - [How It Works](https://www.deep-dive.com/how-deepdive-works) - [Resources](https://www.deep-dive.com/resources) - [About](https://www.deep-dive.com/about) - [Request a Demo](https://www.deep-dive.com/contact) [Log in](https://client.deep-dive.com/login) [Request a Demo](https://www.deep-dive.com/contact) [Log in](https://client.deep-dive.com/login) [Blog](https://www.deep-dive.com/resources/tag/blog) # The anatomy of an LLM response…what compliance professionals need to know is happening under the hood Discover the hidden complexities behind LLM responses and why compliance professionals need to approach AI outputs with caution before making decisions. 4 mins read Share: ![](https://www.deep-dive.com/hs-fs/hubfs/LLM%20and%20human%20working%20together%20shutterstock_2742687765%20(1).jpg?width=520&height=379&name=LLM%20and%20human%20working%20together%20shutterstock_2742687765%20(1).jpg) ### ON THIS PAGE You typed a search subject into ChatGPT, Gemini, or Copilot. The output looked surprisingly thorough, a structured summary that you thought could actually work. But before you use that output as the basis of a compliance decision, it's worth understanding what was happening behind the consumer interface. Because the pipeline that produced it was making a series of invisible decisions — and for compliance purposes, those decisions matter. ##### **What happens between your question and the answer?** When you type a query into ChatGPT or Gemini, you are not talking directly to the AI model. You're using a consumer product built around it. Between your question and the response sits an invisible process — and that process makes a lot of decisions you can't see and can't configure. | **🖥️ What you see** | **🔧  What's actually happening** | | --- | --- | | You → Chat box → Query → Answer with citations | You → API Gateway → Safety Filters → Orchestrator → Prompt Templates → Tool Router → Web Search → Page Fetcher → Content Extractor → Token Truncator → Context Manager → Summariser → Reasoning Model → Output Safety Filter → Citation Formatter → Answer | Every step in that process makes decisions about what to include, what to compress, and what to discard. None of those decisions are visible to you. None were designed with regulatory defensibility in mind. ##### **Data compression and data loss** Every word read by an LLM is processed as a "token", a unit of text roughly equivalent to a word. Models have a limit on how many tokens they can hold in memory at once, this is their context window. When a consumer LLM visits a web page during a research task, it cannot hold the full text within that limit, so it compresses. A several thousand word article about your subject becomes a fraction of its original length. The specific dates, the business associate names mentioned, the corporate relationship referred to later in the source — these can be the tokens that get discarded. For basic research, that loss could be acceptable. For compliance work, those details are often the entire point. ##### **The hidden system prompt** Before the model looks at your query, it receives a set of hidden instructions, called a system prompt that defines how it must behave. In consumer AI tools, this system prompt can run to roughly 20 pages of rules the model must follow before it engages with your question. Some of these rules create friction for compliance work: | **LLM consumer behaviour** | **Why it matters for EDD** | | --- | --- | | Avoids making potentially defamatory statements about real people. | Adverse findings are exactly what EDD exists to surface — the model hedges what it should state clearly. | | Cautious about politically sensitive content involving real people. | PEP investigations require direct reporting of political exposure — caution softens findings that could be facts. | | Not designed to reproduce substantial passages from source material | You cannot verify whether a source actually says what the model claims it says | These rules exist for good reasons in a consumer context. They create friction in a compliance context. The model has been instructed to be cautious about exactly the things you need it to be direct about. The reproduction limit is worth dwelling on. Consumer LLMs are not designed to reproduce substantial passages from source material. If you ask what a court judgment says about your subject, you will get a paraphrase. If you ask what a regulatory notice says, you will get a summary. The passage that supports the finding is not something the tool is built to show you — which means you cannot verify whether the source actually says what the model claims it says. ##### **What this means in practice** The practical consequence is that consumer LLM outputs make poor compliance evidence. They are useful for orientation, for getting a quick picture of a subject before a deeper investigation. They are not suitable as the investigation itself. A report you cannot fully source, produced by a process you cannot describe, through a pipeline you cannot audit, is not a defensible basis for a risk decision. ##### **How DeepDive approaches this differently** DeepDive uses the same underlying AI models, Claude, GPT-4, and Gemini, but without the consumer guardrails. The platform is built specifically for investigative and compliance work: source data extraction tuned for surfacing adverse findings, prompts tuned for compliance objectives rather than consumer caution, every output linked to a verified, archived source. The pipeline is visible, auditable, and designed from the ground up for regulatory defensibility. If you want to try DeepDive in action on one of your cases, email [info@deep-dive.com](mailto:info@deep-dive.com) or book a discovery call with the team. Book a discovery call   [Blog](https://www.deep-dive.com/resources/tag/blog) DEEPDIVE FOR EDD ## Spotting red flags is the easy part. DeepDive automates Enhanced Due Diligence — turning fragmented public data into structured, decision-ready intelligence. [Request a Demo →](https://www.deep-dive.com/contact) Book a 20-minute demo → ![DeepDive-white-logo with strap PNG](https://www.deep-dive.com/hs-fs/hubfs/Brand/DeepDive-white-logo%20with%20strap%20PNG.png?width=260&height=60&name=DeepDive-white-logo%20with%20strap%20PNG.png) ## Why AI-Powered EDD Matters Traditional EDD is slow, fragmented, and resource-intensive. DeepDive's AI-powered EDD cuts through that complexity — automating intelligence gathering, surfacing connections manual review would miss, and turning fragmented public information into structured, actionable intelligence. The result is faster investigations with greater depth and consistency. [Book a demo](https://www.deep-dive.com/contact) See how DeepDive works ## # **Frequently Asked Questions** ## Related posts [![AML compliance exec with too much work to cope with](https://www.deep-dive.com/hs-fs/hubfs/shutterstock_53687581.jpg?height=200&name=shutterstock_53687581.jpg)](https://www.deep-dive.com/resources/how-deepdive-multiplies-edd-capacity-without-additional-headcount) [Blog](https://www.deep-dive.com/resources/tag/blog) ## [How DeepDive multiplies EDD capacity without additional headcount](https://www.deep-dive.com/resources/how-deepdive-multiplies-edd-capacity-without-additional-headcount) AML compliance teams can sometimes be the bottleneck. Not because they lack skill or dedication,... [![AI creating a Body of Knowledge ](https://www.deep-dive.com/hs-fs/hubfs/Digital%20body%20of%20knowledge%20shutterstock_2351612449.jpg?height=200&name=Digital%20body%20of%20knowledge%20shutterstock_2351612449.jpg)](https://www.deep-dive.com/resources/the-body-of-knowledge-from-resolved-entity-to-insight) [Blog](https://www.deep-dive.com/resources/tag/blog) ## [The Body of Knowledge: From resolved entity to insight](https://www.deep-dive.com/resources/the-body-of-knowledge-from-resolved-entity-to-insight) After DeepDive's entity resolution ensures all collected data pertains to the correct individual,... [![](https://www.deep-dive.com/hs-fs/hubfs/0.jpg?height=200&name=0.jpg)](https://www.deep-dive.com/resources/how-deepdive-uses-entity-resolution-to-eliminate-false-positives) [Blog](https://www.deep-dive.com/resources/tag/blog) ## [How DeepDive uses Entity Resolution to eliminate false positives](https://www.deep-dive.com/resources/how-deepdive-uses-entity-resolution-to-eliminate-false-positives) After DeepDive's NLP digests and structures information from hundreds of sources, a critical... [![DD\_Stak\_Full\_white\_text-266px wide](https://www.deep-dive.com/hs-fs/hubfs/DD_Stak_Full_white_text-266px%20wide.png?width=265&height=149&name=DD_Stak_Full_white_text-266px%20wide.png "DD_Stak_Full_white_text-266px wide")](https://www.deep-dive.com) ### Use Cases [Enhanced Due Diligence](https://www.deep-dive.com/enhanced-due-diligence-reports) [False Positive Discounting](https://www.deep-dive.com/enhanced-due-diligence-reports#Falsepositivediscount) [Investigation](https://www.deep-dive.com/investigation-automation)   ### Company [How DeepDive works](https://www.deep-dive.com/how-deepdive-works) [About us](https://www.deep-dive.com/about) [FAQ](https://www.deep-dive.com/enhanced-due-diligence-reports#faqs) [Contact Us](https://www.deep-dive.com/contact)   ### Recent Blogs ###### [What do kitchen knives and LLMs have in common?](https://www.deep-dive.com/resources/what-do-kitchen-knives-and-llms-have-in-common) ###### [What regulators see when your firm uses DeepDive for EDD](https://www.deep-dive.com/resources/what-regulators-see-when-your-firm-uses-deepdive-for-edd) ###### [How DeepDive multiplies EDD capacity without additional headcount](https://www.deep-dive.com/resources/how-deepdive-multiplies-edd-capacity-without-additional-headcount) ![Cyber Essentials Plus logo image\_2025-05-29\_101823.3940220000\_256x256\_q85\_padding\_upscale\_water\_mark](https://www.deep-dive.com/hs-fs/hubfs/Cyber%20Essentials%20Plus%20logo%20image_2025-05-29_101823.3940220000_256x256_q85_padding_upscale_water_mark.png?width=76&height=76&name=Cyber%20Essentials%20Plus%20logo%20image_2025-05-29_101823.3940220000_256x256_q85_padding_upscale_water_mark.png)    [![Linkedin Icon](https://www.deep-dive.com/hs-fs/hubfs/Group%2073-png.png?width=82&height=82&name=Group%2073-png.png)](https://www.linkedin.com/company/deep-dive-com) --- Copyright © 2026 DeepDive ###### All Rights Reserved | [Website Privacy Policy](https://www.deep-dive.com/en-gb/website-privacy-policy) ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Dave", "url" : "https://www.deep-dive.com/resources/author/dave" }, "dateModified" : "2026-04-16T14:21:26.332Z", "datePublished" : "2026-04-16T14:18:05.000Z", "headline" : "The anatomy of an LLM response…what compliance professionals need to know is happening under the hood", "image" : [ "https://www.deep-dive.com/hubfs/LLM%20and%20human%20working%20together%20shutterstock_2742687765%20(1).jpg" ], "mainEntityOfPage" : { "@id" : "https://www.deep-dive.com/resources/the-anatomy-of-an-llm-response-what-compliance-professionals-need-to-know-is-happening-under-the-hood", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://www.deep-dive.com/hubfs/DD_Horizontal%20blue%20on%20white%20SVG.svg" }, "name" : "DeepDive" } } ```