Text-to-SQL with a local LLM lets you ask your ERP or CRM database a question the way you would ask a colleague and get a figure from your own data, without any cloud service. A language model on an office workstation converts the question into SQL, the language databases are queried in; the database runs that query under an account that may only read, and the result comes back as a table or chart along with the query text. The questions, the table structure and the answers stay inside the company. For a controlling or sales department, a sensible place to start is one NVIDIA RTX PRO 5000 Blackwell with 48 GB.
Typical questions look like this: "which customers bought less this quarter than in that quarter last year" or "open orders above a certain amount, split by sales manager". Behind each one sit several tables that have to be joined correctly, plus a filter that counts the way your company counts. Searching documents is a separate job (see the article on RAG over company data): search returns a passage of text, whereas text-to-SQL computes a number, and a faulty filter yields a confident but incorrect figure.
In brief
- The semantic layer matters most: views with readable names, written definitions of key figures and verified example questions.
- On the BIRD benchmark, the highest-ranked single model with open weights gets 74.12% of results right; humans get 92.96% (as of 30.10.2026).
- DB-GPT (MIT) and the LangChain SQL agent (MIT) work with local models, while the Vanna repository was archived in March 2026.
- By our sizing rule, an RTX PRO 5000 Blackwell with 48 GB holds Qwen3.8-27B at 8-bit plus 16 concurrent questions of 10,000 tokens each; the 72 GB variant holds 44.
- Safety comes from database rights: SELECT only on prepared views, a reporting copy, and a log of every query.
How accurately a model writes SQL
BIRD spans 95 databases and counts an answer as correct when the output of the generated query matches the output of a reference query.
| BIRD participant, test set | What is scored | Correct results, % |
|---|---|---|
| Humans: data engineers and students of databases | reference | 92.96 |
| Top of the main ranking (entry of 22.10.2026) | whole system | 83.40 |
| Gemini-SQL2 (Google) | single model, weights not linked | 80.04 |
| Kwai-AutoSQL-32B | single model, weights on Hugging Face | 74.12 |
Source: BIRD leaderboard, status on 30.10.2026.
Even with a strong open model, about one answer in four on a public test is wrong, and company databases tend to be messier than BIRD. The tool still earns its place if you keep three rules:
- display the SQL next to every answer, so a person who knows the data notices a bad table join;
- maintain a library of verified questions with their correct queries, which the tool uses as examples;
- label answers that come from a verified query differently from fresh ones.
The harder Spider 2.0 benchmark is built on enterprise databases that frequently have over 1,000 columns. On its Snowflake set, gpt-oss-120b scored 65.45 with one agent framework but only 13.10 with a different one. In other words, the wrapper around the model (how it sees the schema, checks and repairs queries) weighs as much as the model itself.
We found no published measurements of text-to-SQL accuracy for questions asked in Ukrainian or Russian, so build your test set from the way your staff actually write, in both languages.
A semantic layer: your business terms for the model
The model sees nothing but table and column names. Even clear names do not explain what your company counts as revenue, which order statuses mean cancellation, or which date puts an order into a quarter. A semantic layer fills that gap: a written description of the data in business language, placed between the database and the model. It states what each table means, which joins are permitted and the formula behind each indicator.
WrenAI's context layer shows what such a layer looks like. Its Modeling Definition Language covers models, columns, relationships, metrics, views and cubes, while the plain files instructions.md and queries.yml store lists of allowed values, units, permitted joins and definitions. You can manage without WrenAI too:
- IT creates a few database views (stored queries that behave like tables), named in plain words after the key questions: sales per customer and month, open orders with the responsible manager.
- It adds one explanatory sentence to every column.
- It adds verified sample questions with their correct queries.
From then on the model talks only to the views, and the raw ERP tables stay beyond its reach. The outcome depends on this step more than on the choice of model: twenty definitions and thirty verified questions from a controller give the tool what an experienced colleague knows. Store the layer in Git; then each change to a definition carries a date and gets reviewed.
Read-only access to the ERP and CRM database
The real security boundary is the account the tool uses to log in to the database. LangChain's documentation advises keeping connection rights as narrow as the agent requires. A ban on changing data inside the prompt is just a sentence of text; only the database rights enforce it.
- A copy instead of the live system. Queries are sent to the data warehouse or a replica kept for reporting, and a heavy query then cannot hold up the entry of orders. Check with your ERP vendor whether your support contract permits direct access to its database.
- A dedicated read-only user. SELECT rights on the semantic-layer views and nothing more. In PostgreSQL,
default_transaction_read_onlyonly sets a default that a session can override, so the rights are what count;statement_timeouton this account stops queries that run too long. - Row-level rules. If managers may see only their own customers, that rule lives in the database, and the database must know who is asking: a shared account sees whatever its role allows. So each team gets a separate role and a separate connection in the tool. With PostgreSQL, create views with
security_invoker = true; otherwise the owner's row rules apply rather than those of the person querying. - Checks and a log. One SELECT statement per question, a cap on returned rows, and a log of the user, the question and the SQL.
Free-text fields and customer notes may carry instructions that land in the model's context (prompt injection). What such an instruction can make the agent do is limited by the same database rights.
Customer data in a CRM is still personal data on your own server. Under Law No. 2297-VI, processing needs one of the six grounds in Article 11 (consent, contract, legitimate interest and so on), and people are told the purpose of processing and their rights (Article 12). Your own server removes the outside provider but not these duties, and decisions with legal effects for a person should not be left to the model alone (Article 8 grants protection against automated decisions). Bill No. 8153, which is closer to the GDPR, had not been adopted as of November 2026. This is not legal advice; your lawyer will confirm the details.
Open text-to-SQL tools: DB-GPT, LangChain, WrenAI, Vanna
The status of these projects shifted during 2026, so check it before you start.
| Tool | Licence | Local model | Status on 30.10.2026 |
|---|---|---|---|
| DB-GPT | MIT | vLLM, llama.cpp, OpenAI-compatible servers | active, last release in June 2026 |
| SQL agent (LangChain) | MIT | Ollama and other providers through integrations | framework, maintained; no ready app |
| WrenAI | Apache 2.0 core, commercial interface | none named in the README | not archived, open part aimed at AI agents |
| Vanna | MIT | Ollama listed | archived 29.03.2026, read-only |
Source: GitHub repositories and READMEs (Vanna, WrenAI, DB-GPT, LangChain), DB-GPT releases page, LangChain guide "Build a SQL agent" and the ChatOllama page; read on 30.10.2026.
DB-GPT, under MIT, presents itself as an "open-source agentic AI data assistant": it accepts natural-language questions, writes SQL, and produces charts, dashboards and reports. The web interface listens on port 5670. The README is silent on user accounts and permissions, so access control goes in front of it: corporate sign-in or an authenticating proxy, as for the chat front ends in the article on a private ChatGPT alternative.
The LangChain SQL agent is a guide with building blocks for developers rather than a ready application. The agent reads the list and structure of tables, drafts a query, lets the model review it, runs it, fixes it after an error and replies. Its system prompt forbids DML statements (INSERT, UPDATE, DELETE, DROP and the like), and HumanInTheLoopMiddleware can halt the agent before each query until a person approves it. The guide's examples rely on cloud models, but the ChatOllama integration lets a local model call tools.
WrenAI calls itself GenBI for AI agents, with governed text-to-SQL through an open context layer. The engine, SDK and skills are open under Apache 2.0, whereas row- and column-level security, user management, the GenBI interface, dashboards and embedding are marked commercial.
Vanna ("Chat with your SQL database") reached version 2.0 with a ready chat component, but its owner archived the repository on 29 March 2026. It will get no more fixes, so it is not a good choice for new projects.
Which local model writes the SQL
There are two types of model. SQL-tuned models, for example Kwai-AutoSQL-32B with Apache 2.0 weights, or Arctic-Text2SQL-R1-7B from Snowflake (Apache 2.0; its card reports 68.5 on BIRD's test set), convert a question plus the table definitions into one query. Agents like DB-GPT or the LangChain SQL agent need a general-purpose model with tool calling: gpt-oss-20b, gpt-oss-120b (both Apache 2.0) or Qwen3.8-27B (released August 2026 under Apache 2.0), whose card gives no SQL benchmark. Nobody has published a comparison of these models on schemas like yours, so your own test decides.
Qwen3.8-27B reasons before answering: its card sets the highest reasoning level by default, and for SQL steps the medium or low level cuts the wait. The speed of a single answer depends on memory bandwidth and on how much data has to be read for each token. The RTX PRO 5000 offers 1,344 GB/s, and a 27B model at 8-bit reads about 28 GB per token, so by our estimate the ceiling is roughly 47 tokens/s. An agent writing 500 tokens in each of four steps will need at least about 43 seconds (again our estimate).
Which GPU you need for text-to-SQL
We use our own rule: 90% of the memory the driver reports, minus 3 GiB per card, minus the model weights; what remains is for questions being processed. A token is a word or part of one. In this example a single question, with instructions, selected views, examples and the agent's steps, takes about 10,000 tokens. We assume a 16-bit cache, which is what Ollama keeps by default.
| Tier | Model (weights) | Card | Concurrent questions | Suits |
|---|---|---|---|---|
| Starter | gpt-oss-20b (12.8 GiB) | RTX PRO 4000 Blackwell, 24 GB, one card | 24 | a single team with a handful of views |
| Main | Qwen3.8-27B, 8-bit (27.9 GiB) | RTX PRO 5000 Blackwell, 48 GB, one card | 16 | controlling and sales |
| Headroom | Qwen3.8-27B, 8-bit (27.9 GiB) | RTX PRO 5000 Blackwell, 72 GB, one card | 44 | several departments |
| Upper | gpt-oss-120b (60.8 GiB) | RTX PRO 6000, 96 GB, one card (Workstation Edition or Max-Q) | 64 | larger model for the whole company |
Source: our estimate by the sizing rule (usable memory 18.5, 40.0, 61.5 and 83.0 GiB); weights from Ollama tags (qwen3.8 q8_0, 30 GB) and the gpt-oss files on Hugging Face; 16-bit cache per token: 24 KiB for gpt-oss-20b, 64 KiB plus 0.14 GiB fixed state for Qwen3.8-27B, 36 KiB for gpt-oss-120b; an FP8 cache in vLLM fits more questions; not measured.
Questions in progress are not the same as users: ten managers seldom ask in the very same second, so each figure serves a bigger team. A BI team on a large data warehouse needs a server, with H200 NVL or RTX PRO 6000 Server Edition cards. General memory formulas are collected in the article on GPU memory for LLMs. ETE works out the configuration for your database type, number of tables and users from the actual load.
A pilot on one subject area
Gather 50 to 100 genuine questions from the future users, and have someone who knows the data confirm the right answer to each. Build the views and definitions for those questions, run them through every candidate model in one tool, then tally right answers, wrong answers and refusals. Rerun the test whenever the model, prompt or views change.
A pilot on one area, such as sales, shows whether the answers can be trusted before other areas are added. Moving from one workstation to a shared server for several departments is covered in the article on sizing a private ChatGPT server by company size. Vixen.UNO engineers can take on installing the model, DB-GPT and the read-only database access.
FAQ
Is Vanna AI still maintained?
No: the vanna-ai/vanna repository on GitHub was archived on 29 March 2026 (as of 30.10.2026). The MIT code remains available, but there will be no fixes. Maintained alternatives that work with local models are DB-GPT and the LangChain SQL agent.
How accurate is text-to-SQL?
On BIRD, humans get 92.96% of results right, the highest-ranked single model with open weights 74.12%, and the top system of the main ranking 83.40% (as of 30.10.2026). Display the generated SQL with every answer and maintain a library of verified questions.
How can I chat with my database without a cloud service?
Run the model on your own workstation with vLLM or Ollama, point a text-to-SQL tool such as DB-GPT or the LangChain SQL agent at it, and let the tool log in to the database with an account that can only read. You type the question, the model drafts SQL, the database executes it and the reply shows the query beside the figures. Check accuracy for questions in Ukrainian or Russian on your own pilot set.
Which GPU do I need for text-to-SQL with a local model?
By our sizing rule, an RTX PRO 4000 Blackwell (24 GB) serves gpt-oss-20b to 24 concurrent questions, a 48 GB RTX PRO 5000 holds Qwen3.8-27B at 8-bit for 16 at once, and the 72 GB variant manages 44. A BI team on a large warehouse needs a server.
Can the AI change or delete data in the database?
No, as long as its account may only read. Grant SELECT on chosen views, query a reporting copy, and refuse every generated statement other than one SELECT. A ban written into the prompt is not protection by itself.
What ETE supplies for this
- NVIDIA RTX PRO 5000 Blackwell 48 GB: Qwen3.8-27B at 8-bit and 16 concurrent questions for controlling and sales.
- NVIDIA RTX PRO 4000 SFF Blackwell 24 GB: the same 24 GB as the entry tier in the calculation: gpt-oss-20b and up to 24 concurrent questions from one team.
- NVIDIA RTX PRO 6000 Blackwell Max-Q 96 GB: gpt-oss-120b and about 64 concurrent questions from across the company.
- ETE AI Workstation: a workstation with a 96 GB RTX PRO 6000 and 256 GB of ECC memory for the model, DB-GPT and the query log.
Need a workstation for questions to your database?
Tell us which database you run, how many tables it has, how many people will ask, and ten typical questions from your team in the languages they write them in. ETE will work out the configuration and, if needed, Vixen.UNO engineers will install the model and DB-GPT and set up read-only access.