When enterprise RAG on company documents gives a wrong answer, the cause is usually search: the passage it needed never reached the model. Such a project (RAG, retrieval-augmented generation, where the model answers from documents found for it) is therefore built around search. Access rights are checked at the moment of retrieval, documents are split along their structure, quality is measured on real questions, and the index is kept current.
A Ukrainian company has one more question to settle: documents in Ukrainian, Russian and English sitting in one corpus. The most recent open data on this comes from the UNLP 2026 competition, in which systems looked for answers to Ukrainian questions in roughly 240 PDFs with no internet access. Its results help with the choice of embedding models.
In brief
- Search determines RAG quality: if the right passage is not among those found, neither the model nor the prompt can rescue the answer.
- Access rights are applied as a filter during search; an instruction in the prompt does not control access.
- Before tuning anything, write down 50 to 100 real questions together with the correct answer and the source document.
- At UNLP 2026, the KPI team got its top result with Qwen3-Embedding-8B and Qwen3-Reranker-8B (both Apache 2.0); reranking lifted Recall@1 from 0.6957 to 0.7935.
- By our estimate from NVIDIA NIM figures, a single L40S running NVIDIA's 1B embedding model turns 300,000 documents of ten 512-token chunks each into vectors in about 5.8 hours.
How RAG works: four steps
A RAG pipeline has four stages:
- locate passages in the company's documents that answer the question;
- insert them into the prompt;
- ask the model to answer from those passages only;
- show the user where the answer was taken from.
Most of the effort goes into the first stage. When the retrieved passages contain the answer, a modern instruction-tuned model will normally produce it, though it picks up an answer buried mid-way through a long context less reliably. When they do not contain it, swapping the prompt or the model will not help. So measure search first, and only then edit prompts or change the model.
For planning, this means RAG is largely data engineering work, and people and timelines should be budgeted accordingly. Ready-made apps for chatting with documents are reviewed in the article on a private ChatGPT alternative, while figures from an ERP or CRM call for text-to-SQL, covered in the article on asking your database questions in plain language.
Access rights in RAG: filter during retrieval
A leak of restricted documents has legal consequences, so that is where to begin. The tempting shortcut is to pour everything into one collection and instruct the model to keep back any document the user is not entitled to. That is no access control. An instruction in a prompt only asks a probabilistic system for something; when the model reads it, the payroll spreadsheet is already sitting in the context window.
Rights are applied at the search stage, so search never returns a passage the user could not open in the source system. Three measures make this work.
- Access lists (ACLs) in the index. Each chunk carries the list of users and directory groups permitted to read its document, and each query is filtered by the identity and groups of whoever is asking. With an approximate vector index, the filter must run before or during the search; if it is applied only to candidates afterwards, someone with access to a small slice of the corpus ends up with too few passages.
- Resync when rights change. A person moves to another group: the documents stay the same, yet their access is different. Indexes that only watch when files were last modified overlook this, and outdated rights linger until the document itself is edited.
- A log of what was found for whom. It answers the question whether the assistant could have shown a given document.
Filtering during search keeps other people's documents out of the context, but it does not stop a document the user is allowed to read from carrying instructions for the model, for instance to leak data via a tool call or a link. That technique is called indirect prompt injection; OWASP's Top 10 for LLM Applications 2026 lists prompt injection first, under LLM01.
Where source systems have tangled inherited rights (the classic case is a document management system that has grown for years), this step sets the project timeline, and it is better to find that out at the outset. Some products carry rights over on their own: Onyx's paid Enterprise Edition syncs them from SharePoint, Teams and Confluence. A payroll sheet or customer records contain personal data, which under Law No. 2297-VI may be processed only on one of the grounds in Article 11 and for a defined purpose. A local server lowers the risks of data transfer but replaces neither the legal ground nor data protection.
How to chunk documents for RAG
Before indexing, documents are cut into chunks, and the way they are cut decides what search can find. First, though, you need clean text and tables out of the PDFs, and Cyrillic is a topic of its own. Docling does not recognise text itself and relies on engines: Ukrainian comes from RapidOCR (models cyrillic, eslav) or EasyOCR, and Tesseract has ukr and rus language data. At UNLP 2026 the KPI team dropped Docling because it was slow and crashed on some PDFs, and switched to pymupdf4llm for files with a text layer. More in the article on PDF parsing for RAG.
| Mistake | Result | Better approach |
|---|---|---|
| Fixed-length pieces cut at random points | a definition is split in two and neither half answers the question | split by structure (headings, sections, list items) with a modest overlap |
| Oversized chunks | search returns whole pages, the key sentence gets lost, costs go up | one complete idea per chunk; fetch more chunks instead of longer ones |
| Undersized chunks | the passage loses its subject: "replace every year", but replace what | attach the parent heading to every chunk |
| Tables merged into running text | numbers lose their row and column labels | keep each table whole, headers included |
| No metadata | nothing can be filtered or dated; a withdrawn policy sits next to the current one | record source, section, date, version and owner of the document |
Source: our summary of common document-splitting mistakes in RAG projects.
Metadata pays off twice. During search it lets you narrow the query to a document type or a period. In the answer, a citation with the real document title and date is what makes the system credible: the user opens the source and checks the answer within seconds.
Hybrid search (keywords plus vector similarity) is worth the modest extra work in nearly every corporate corpus. Embeddings capture meaning yet can miss an exact match, so part numbers, contract numbers, error codes and project names also need keyword search.
Embeddings for Ukrainian: multilingual RAG
We did not find a separate Ukrainian section in MTEB or MMTEB in primary sources. UNLP 2026 offers a practical reference point: Ukrainian questions over about 240 PDFs, offline.
| Model | Licence | What is known for Ukrainian |
|---|---|---|
| Qwen3-Embedding-8B | Apache 2.0 | KPI system at UNLP 2026: 0.9426 / 0.9592 (public / private); 32k-token context |
| Qwen3-Embedding-0.6B | Apache 2.0 | UNLP 2026: 0.9370 / 0.9507 |
| Qwen3-Reranker-8B | Apache 2.0 | reranking lifted Recall@1 from 0.6957 to 0.7935 |
| bge-m3 (BAAI) | MIT | 8,192 tokens; no Ukrainian figures on the card |
| multilingual-e5-large | MIT | 512 tokens; uk and ru evaluated in CrossNews-UA |
| jina-embeddings-v3 | CC-BY-NC-4.0 | uk and ru among its 30 tuned languages; non-commercial licence |
Source: arXiv 2605.10296 (UNLP 2026 shared task), model cards on Hugging Face, arXiv 2510.19628 (CrossNews-UA); read on 13.11.2026.
The UNLP figures measure the accuracy of the whole system (search, reranker and generator together), not of an embedding model alone. They show only a small gap between the 8B and 0.6B variants, so test the smaller model first on your own question set.
A multilingual embedding model finds meaning across a language boundary, but keyword search matches only within a single language. Store each document's language as metadata, so that results can be balanced or filtered, and make sure the evaluation set contains questions in every language of the corpus. For generating answers in Ukrainian, the Ukrainian LLM Leaderboard (lang-uk, May 2026) gives the highest scores to Gemma 4 26B A4B in reasoning mode (Apache 2.0); among models fine-tuned on Ukrainian there is MamayLM v2.0 at 12B and 27B (Gemma Terms of Use).
How to evaluate RAG: a question set and a pilot
Collect between fifty and a hundred genuine questions from future users, each paired with its correct answer and the document that should supply it. It takes a domain expert, and no other task in the project pays off as much. Without such a set, every change is a matter of taste, and nobody can tell whether a retuned split helped. With the set you measure two things:
- search hit rate: how often the right passage is among those retrieved. If it is low, no later step can rescue the answer, and it is clear where the work lies;
- answer correctness: how often the answer is right once the needed passage has been found. Only when search is good and this figure is low does the problem sit in the prompt or the model.
A low hit rate points to splitting, metadata or search; low correctness despite good hits means the prompt or the model is at fault. One overall score for the final answer hides which part broke.
Begin with a single department, a single set of documents and one well-defined kind of question, and bring future users in from day one. Bring the hit rate up on this set and only then add another corpus. Projects get stuck when they index everything, demo it to everyone, gather unrelated complaints and never learn which search problem to fix first. With a narrow pilot, one team gets answers that can be measured, and the pilot's figures then justify the choice of the next set of documents. Vixen.UNO engineers can take on deploying Qdrant, the embedding models and the language model.
Keeping the index current, and when the assistant should decline
Nothing harms the system's reputation more than a confident, well-phrased answer drawn from a document that was replaced four months ago.
- Index on a change event instead of a timetable, if the source system supports it: a corrected policy becomes searchable the same day.
- Put the document date into each citation, so a reader sees when the answer rests on something old.
- Delete replaced documents from the index instead of counting on ranking: if the old version is still there, it will be found sooner or later.
Next to this sits the ability to decline. If nothing relevant above a confidence threshold turns up, the correct reply reads "this was not found in the available documents". Such a reply sends the user to the source system, whereas a confident wrong answer may well be acted upon.
What hardware RAG needs: embeddings, answers, the vector index
The load has two parts: turning the corpus into vectors, and answering questions. Vectorisation is a one-time peak: each document is processed once, after that only changed ones, until you replace the embedding model and everything must be recomputed.
| Card | Texts of 512 tokens per second | 300,000 documents of 10 chunks |
|---|---|---|
| L40S | 144.3 | about 5.8 h |
| L4 | 49.2 | about 17 h |
Source: NVIDIA NIM 1.14.0 documentation of 30.09.2026, NVIDIA 1B embedding model in FP8; processing time by our estimate.
Large embedding models derived from LLMs need days for a corpus like this, and PDF parsing often takes longer than the vectorisation itself. Afterwards the load almost disappears.
Answering is the steady load, and it depends on the number of simultaneous conversations rather than on corpus size. A vector index is measured in gigabytes: roughly 4 bytes × dimensions × number of chunks plus index overhead, or less with quantised vectors. For example, 3 million chunks with 1,024-dimensional vectors take about 12.3 GB before overhead (our calculation). The index usually sits in RAM, except where the database stores it on disk. GPU memory goes to the KV cache for concurrent users, and RAG prompts are long because each one carries retrieved passages; the formulas are in the article on GPU memory for LLMs. The Lapa LLM tokeniser has 80,000 of its 250,000 tokens replaced with Ukrainian ones, so with this model Ukrainian text needs about 1.5 times fewer tokens, and the same prompt less cache.
For most internal deployments this means one server, and a multi-node cluster should follow from measured load. A pair of RTX PRO 6000 Server Edition cards fits pilots and RAG assistants running models of up to roughly 70B parameters; four fit production use by several teams and models. ETE builds both configurations to the task and sizes them from the document count and concurrent users.
FAQ
Do we have to fine-tune a model on our data?
Usually not. Fine-tuning teaches style and format, while facts come from search. For questions like "what does our policy say", RAG is the right fit, with one decisive advantage: update the document and the answer updates too, with no retraining.
How do you respect document access rights in RAG?
Filter during search by the identity of the person asking, so restricted passages are never returned at all. Telling the model to stay discreet does not control anything: at that point the content is already in the context window.
How many documents does RAG need?
Volume matters less than whether people currently lose time searching. A few thousand documents that are searched every day make a stronger case for a project than a million that nobody reads.
Why does the assistant do well in a demo and poorly in daily work?
Demos use questions whose answers are easy to find. Real questions are full of in-house vocabulary, abbreviations and part numbers. The evaluation set exists to close exactly this gap, which is why its questions are collected from real users and not invented by the project team.
Does RAG handle documents in Ukrainian, Russian and English at once?
Yes, with a multilingual embedding model and the document language stored in metadata. At UNLP 2026 the KPI team reached its top result on Ukrainian questions with Qwen3-Embedding-8B and Qwen3-Reranker-8B. Keyword search works only within one language, so the evaluation set has to cover every language in the corpus.
What hardware does RAG on company data require?
Usually one server. Vectorising the documents is a one-time peak, and the steady load comes from how many people ask at once, regardless of collection size. A server with two RTX PRO 6000 Server Edition cards handles pilots and assistants on models up to roughly 70B parameters; with four it serves several teams and several models.
What ETE supplies for this
- ETE AI Server RTX PRO: four 96 GB RTX PRO 6000 Server Edition cards (384 GB of GPU memory), the four-card configuration from this article for several teams and models.
- NVIDIA RTX PRO 6000 Server Edition 96 GB: the basis of a two-card configuration for a pilot with models up to about 70B.
- NVIDIA L40S 48 GB: corpus vectorisation, 144.3 texts of 512 tokens per second by NIM figures, or 300,000 documents in about 5.8 hours.
Planning an assistant on your company documents?
Tell us how many documents you have and in which languages, which systems hold them, how access is granted there and how many people will ask at the same time. ETE will work out the server configuration and, if needed, Vixen.UNO engineers will set up Qdrant, the embedding models and the language model.