Document Intelligence and RAG for Compliance: What Production Actually Looks Like

Compliance teams in Indian enterprises are drowning in unstructured documents. RBI master directions and circulars, SEBI consultation papers and operational instructions, IRDAI circulars, MCA filings, internal policies, audit trails, contracts, GST notices, regulatory responses. The volume is large, the change velocity is high, and the cost of missing something material is increasingly punitive.

Retrieval-augmented generation is the obvious AI category to apply to this work. RAG, when built well, allows compliance teams to query a corpus of documents in natural language, get answers grounded in the source material, and follow the citation back to the exact paragraph. The pattern is well-suited to compliance because answers must be verifiable — and RAG, unlike pure LLM generation, provides verifiability by design.

The honest reality is that most RAG deployments in Indian enterprises are toy demos. They work for the showcase questions, they fail on the actual compliance questions, and they get quietly shelved within six months. The failure modes are predictable, the fixes are known, but most teams discover them only after the first iteration disappoints.

---

What Document Intelligence Means in a Compliance Context

Document intelligence is a broader category than RAG. In a compliance context, it covers four distinct activities.

Extraction. Pulling structured information out of unstructured documents — vendor name and date from an invoice, counterparty and clauses from a contract, policy applicability from a regulatory notification. Mature, well-served by combinations of OCR, layout-aware models, and LLM-based extractors.

Classification. Categorising documents by type, topic, jurisdiction, and priority. A new RBI circular needs to land in front of the right people within the bank within hours, not weeks.

Comparison. Detecting changes between two versions of a document, comparing a contract against a template, comparing a regulatory position against the enterprise's current practice. The work that historically consumed legal associate hours.

Question answering. The RAG core. Given the corpus, answer questions grounded in source material with citation. This is where the most attention sits, and where the most failures happen.

A real compliance AI capability does all four. A RAG-only capability is incomplete and will be felt as incomplete by the compliance team within weeks of use.

---

Where RAG Genuinely Pays Back

The high-value RAG applications in Indian compliance work cluster in five places.

Regulatory change management. Indian regulators — RBI, SEBI, IRDAI, MeitY, MCA, GST authorities, CERT-In — publish circulars, notifications, master directions, consultation papers, and FAQs at a pace no compliance function can manually track in real time. A RAG system over the corpus, with a monitoring layer that watches for new publications and an analysis layer that summarises implications for the enterprise's specific obligations, is a force multiplier. The compliance team's role shifts from manual scanning to reviewing flagged items.

Contract review. Comparing an inbound contract against the enterprise's standard template, flagging deviations, identifying high-risk clauses, suggesting redlines. Works particularly well in B2B sales contracts, vendor master agreements, NDAs, and partnership agreements. The first-pass review is automated; the senior reviewer focuses on genuinely substantive issues.

Audit preparation and response. The audit team asks for evidence of a particular control, a particular policy, a particular transaction trail. A RAG system over the relevant document corpus, with appropriate citation, dramatically compresses the time to evidence.

Internal policy and SOP Q&A. Employees asking what the policy is on a specific scenario — travel, expenses, conflicts of interest, data handling. A RAG system over the policy library replaces a substantial volume of compliance helpdesk traffic.

KYC document verification. Extraction, validation, and cross-referencing of identity documents, address proofs, business documents, and beneficial ownership disclosures against the regulatory requirements and the enterprise's risk profile.

The common thread across these five: defined corpus, verifiable answer, serious cost to getting it wrong.

---

The Architecture

A production-grade RAG system for compliance has more components than the typical introductory diagram suggests. Each component is a place where the system can fail.

Ingestion. Documents arrive as PDFs (scanned and digital), Word documents, emails, web pages, regulatory portal feeds, internal system exports. The ingestion layer normalises formats, extracts text, preserves structure — headings, tables, lists — and captures metadata (source, date, jurisdiction, version). Tables are handled specifically because tables are where compliance information often lives.

Chunking. Naive chunking by character count loses the relationship between a regulatory clause and its surrounding context, breaks tables, and separates definitions from their usage. Structure-aware chunking that respects document headings, paragraphs, and tables produces materially better retrieval. For regulatory documents, chunking by clause or section with overlapping context windows is the working pattern.

Embedding. For Indian compliance work, English-only embedding models work well for English regulatory text. Multilingual embedding models are needed when the corpus includes vernacular documents. The embedding model should be benchmarked on the enterprise's own corpus, not on generic benchmarks.

Retrieval. Pure vector similarity often retrieves chunks that are topically related but not directly answering. Hybrid retrieval combining vector similarity with keyword search (BM25) works substantially better in practice. Filter and metadata constraints — jurisdiction, date range, document type — are essential for compliance use cases.

Reranking. A second-stage model that reorders retrieved candidates by relevance. Adds latency, materially improves retrieval quality. For compliance use cases where retrieval quality matters more than latency, reranking is standard.

Generation and citation. The LLM produces the answer given the retrieved chunks. The most important rule: the LLM must be instructed not to answer outside the retrieved context and to refuse when the context does not contain the answer. Every answer must point back to the specific source chunks and underlying source documents. This is not optional in compliance.

Evaluation. A continuous evaluation harness that measures retrieval quality, answer quality, citation accuracy, and refusal behaviour on a ground truth question set.

---

The Citation Requirement

The single most important characteristic of a compliance RAG system is non-negotiable citation. Every answer must point to the specific source paragraph or clause, verifiable in one click.

The reasons are operational and legal.

Operationally, the compliance professional reading the answer will not, and should not, trust the LLM. The answer is a starting point. Verification means reading the cited source. If the citation is wrong, the system is worse than useless because it accelerates a mistake.

Legally, when a compliance position is challenged — by a regulator, by an auditor, by an internal review — the source must be producible. An answer that says "the rule is X" without a source is not a compliance position. It is a guess.

For trust building: a RAG system that cites well is one the compliance team will use and improve. A system that produces fluent but uncited answers is one they will reject within weeks, regardless of how good the answers actually are.

---

Evaluation: The Discipline Most Teams Skip

A RAG system that has not been evaluated systematically is a guess. Demo-time impression is not evaluation.

A working evaluation discipline has four components: a ground truth question-answer set curated by domain experts, retrieval evaluation tracking whether the correct source chunks are surfaced, answer evaluation assessing factual correctness and citation quality, and regression testing ensuring every system change is validated against the ground truth before deployment.

This discipline is the difference between a RAG system that compounds in usefulness and one that quietly degrades. It is unglamorous work, and it is what separates production-grade from demo.

---

Why This Is a Textbook On-Premise Workload

The documents that matter to compliance are, by definition, the documents the enterprise most needs to control. Contracts. Internal policies. Audit trails. Customer KYC. Regulatory correspondence. Legal positions.

Sending these documents to a cloud LLM API, even with strong contractual data handling, is a position most compliance functions are quietly walking away from.

Data residency rules mean that personal data in KYC documents, customer correspondence, and employment files is governed by DPDP. Contract terms, vendor pricing, and litigation positions are competitive assets. Regulators and auditors increasingly ask where data was processed and with what controls — an on-premise architecture answers these questions cleanly.

The technology required for on-premise compliance RAG has matured to the point where this is no longer a heroic engineering effort. Open-source embedding models, open-source LLMs, open-source vector stores, and open-source orchestration frameworks combine into a stack that runs on a modest GPU footprint and serves a real compliance workload.

Setidure's Granthik platform is built exactly for this layer: document intelligence, OCR, semantic search, structured extraction, and RAG over enterprise document corpora — deployed on-premise, keeping sensitive content where it belongs.

---

What This Means in Practice

Document intelligence and RAG are among the highest-payback AI categories for Indian compliance functions. The applications are real, the technology is mature, the citation requirement is satisfiable, and the on-premise economics work at the scale most enterprises already operate.

The work to do it well is more substantial than the demos suggest. Production RAG has a full architecture, an evaluation discipline, and an operational rigour that the introductory tutorials do not capture. The teams that build all of it ship a system that compounds in value as the corpus grows and the questions evolve. The teams that ship the demo version produce something that fails in six months.

Keep the corpus inside the enterprise. Build the full RAG stack — structure-aware chunking, hybrid retrieval, reranking, strict citation. Invest in evaluation as a first-class capability. Use open-source models where they are good enough. Treat document intelligence as a platform, not a project.

The enterprises that build this discipline will operate compliance functions with a leverage that traditional functions cannot match, and they will hold up to the regulatory scrutiny that is coming.

Reach out to admin@setidure.com to discuss document intelligence and RAG for your compliance or legal function.