Marc Hildmann

Small Models, Real SQL: Text-to-SQL on a Mac with SQLite

A 0.6B model, a 24.7 MB database file and a terminal. Twelve questions, seven right answers, and a clear picture of where small models earn their place.

Most text-to-SQL demos run on a model with hundreds of billions of parameters behind an API. I wanted to know what happens at the other end of the scale: a model small enough to load in 2.26 seconds on a laptop, asked to write queries against a real SQLite file, with nothing leaving the machine.

The model is distil-qwen3-0.6b-text2sql from Distil Labs. The database is Northwind, the classic sample shop data, in the SQLite version by jpwhite3. The machine is my MacBook Pro with an M1 Pro and 16 GB of memory, running macOS 27.2. Everything below was run on it on November 20, 2026, and every number is a measurement, not a vendor claim.

What small models can do locally

A 0.6B model is not a smaller ChatGPT. It is closer to a well-trained reflex: it knows one job and does it fast. Distil Labs took Qwen3-0.6B and fine-tuned it on about 10,000 synthetic text-to-SQL examples generated by DeepSeek-V3. On their own evaluation, it reaches 74% on an LLM-as-a-judge check, against 36% for the untuned base model and 76% for the 685B teacher. That is their benchmark, on their data. Mine is below.

What a model this size gives you on a laptop:

  • Speed. Answers in well under a second, with no network round trip.
  • Privacy by construction. The schema, the question and the data never leave the machine.
  • Cost. No tokens to pay for, no rate limits, no API key to rotate.
  • Predictability. Same input, same output (with temperature 0), on a file you control.

What it does not give you is judgment. Keep that in mind for the limits section.

Why SQLite is a powerful local database

SQLite is a database engine that lives in a library, not a server. A database is one file. There is no daemon to start, no port, no user management. macOS ships the sqlite3 command-line shell in /usr/bin, so it is already on your Mac.

That makes it a natural partner for a local model:

Property Why it matters here
One file northwind.db is 24,702,976 bytes. Copy it, back it up, hand it to someone.
Zero setup sqlite3 northwind.db and you are in.
Real SQL Joins, window functions, CTEs, strftime for dates. Enough for real questions.
Fast on a laptop The heaviest query in my test (revenue across 609,283 order lines) took 410.5 ms.
Schema as text .schema Orders prints the exact DDL the model needs as input.

The last point is the one people underestimate. A text-to-SQL model needs the schema in its prompt, and SQLite hands it to you verbatim. I wrote more about running SQLite seriously in SQLite in production.

The Northwind file I used is a larger variant of the original: 93 customers, 77 products, 16,282 orders and 609,283 order lines, with order dates from 2012 to 2023.

How to get started on macOS

You need Homebrew and at least 2 GB of free disk space.

brew install llama.cpp uv
curl -L -o northwind.db \
  https://github.com/jpwhite3/northwind-SQLite3/raw/main/dist/northwind.db
sqlite3 northwind.db ".tables"

uv is a fast Python package runner. You need it for two things: downloading the model and, if you choose llama.cpp, converting it.

The tools: MLX or llama.cpp

There are two good ways to run a small model on Apple Silicon.

MLX (mlx-lm) llama.cpp
Made by Apple's machine learning research team ggml community
Model format Hugging Face safetensors, as published GGUF
Conversion needed No Yes, for this model
Good for Trying a model in one command A local HTTP server, scripting, other platforms

One detail cost me time, so here it is up front: the model card says a quantized GGUF file is "included in this repository", and there is a Modelfile for Ollama that points to ./model.gguf. The file is not there. As of November 20, 2026 the repository contains only model.safetensors (1,192,135,096 bytes, bfloat16). So with llama.cpp or Ollama you convert it yourself. MLX loads it as is.

Path 1: MLX, no conversion

uv run --with mlx-lm mlx_lm.generate \
  --model distil-labs/distil-qwen3-0.6b-text2sql \
  --system-prompt "$(cat system-prompt.txt)" \
  --chat-template-config '{"enable_thinking": false}' \
  --max-tokens 200 \
  --prompt "Now for the real task, solve the task in question block.
Generate only the solution, do not generate anything else
<question>Schema:
$(sqlite3 northwind.db '.schema Products')

Question: How many products are discontinued?</question>"

The first run downloads the model. The answer:

SELECT COUNT(*) FROM Products WHERE Discontinued = '1';

That is correct, and less trivial than it looks: Discontinued is a TEXT column holding '0' or '1', and the model read that from the DDL.

system-prompt.txt holds the system prompt from the model card. Use it verbatim. The model was trained with it, and the odd XML-ish wrapper in the user message is part of the format:

You are a problem solving model working on task_description XML block:
<task_description>You are given a database schema and a natural language question. Generate the SQL query that answers the question.

Input:
- Schema: One or two table definitions in SQL DDL format
- Question: Natural language question about the data

Output:
- A single SQL query that answers the question
- No explanations, comments, or additional text

Rules:
- Use only tables and columns from the provided schema
- Use uppercase SQL keywords (SELECT, FROM, WHERE, etc.)
- Use SQLite-compatible syntax</task_description>
You will be given a single task in the question XML block
Solve only the task in question block.
Generate only the answer, do not generate anything else

enable_thinking: false matters. Qwen3 models can emit a <think> block first; this one was trained to answer directly.

Path 2: llama.cpp, with a one-time conversion

Download the model, then convert it with the script from the llama.cpp source tree. Match the source to your installed build (llama-cli --version prints it; mine was build 10809):

uvx --from huggingface_hub hf download distil-labs/distil-qwen3-0.6b-text2sql \
  --local-dir distil-qwen3-0.6b-text2sql

git clone --depth 1 --branch b10809 https://github.com/ggml-org/llama.cpp.git

uv run --python 3.12 \
  --with-requirements llama.cpp/requirements/requirements-convert_hf_to_gguf.txt \
  --with-editable llama.cpp/gguf-py \
  python llama.cpp/convert_hf_to_gguf.py distil-qwen3-0.6b-text2sql \
  --outtype q8_0 --outfile distil-qwen3-0.6b-text2sql-q8_0.gguf

The conversion took 16.04 seconds. The Q8_0 file is 639,446,880 bytes, 53.6% of the original. Then start a local server with an OpenAI-compatible API:

llama-server -m distil-qwen3-0.6b-text2sql-q8_0.gguf --jinja --port 8080 -c 4096

First steps: ask the database a question

With the server running, this small script turns a question into SQL and runs it. It pulls the DDL for the tables you name straight from the database file, so the model always sees the real schema:

#!/usr/bin/env bash
# Usage: ./ask.sh northwind.db "question" Table [Table ...]
set -euo pipefail
db="$1"; question="$2"; shift 2

schema=""
for table in "$@"; do
  schema+="$(sqlite3 "$db" ".schema \"$table\"")"$'\n\n'
done

user="Now for the real task, solve the task in question block.
Generate only the solution, do not generate anything else
<question>Schema:
${schema}
Question: ${question}</question>"

sql=$(jq -n --rawfile system system-prompt.txt --arg user "$user" '{
    messages: [{role: "system", content: $system}, {role: "user", content: $user}],
    temperature: 0,
    chat_template_kwargs: {enable_thinking: false}
  }' \
  | curl -s http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d @- \
  | jq -r '.choices[0].message.content')

echo "$sql"
echo "---"
sqlite3 -header -column "$db" "$sql"
$ ./ask.sh northwind.db "How many customers are based in Germany?" Customers
SELECT COUNT(*) FROM Customers WHERE Country = 'Germany';
---
COUNT(*)
--------
      11

Round trip including SQLite: 0.43 seconds.

Notice that you choose the tables, not the model. That is deliberate. The model card says it works best with one or two table definitions, and my results agree.

Twelve questions against Northwind

I wrote twelve questions, from simple counts to comparisons across years, plus a hand-written reference query for each. A small script ran the model (via MLX, temperature 0), executed both queries and compared the results. Then I read every miss myself, because "different result" and "wrong" are not the same thing.

# Question Tables Verdict
1 How many customers are based in Germany? 1 Correct (11)
2 Which five products have the highest unit price? 1 Correct, with extra columns
3 How many products are discontinued? 1 Correct (8)
4 How many orders were placed in 2016? 1 Correct (1,506)
5 Average freight cost per shipper ID? 1 Wrong: invented the column ShipperID
6 Which customer company placed the most orders? 2 Valid SQL, misleading answer
7 Orders per employee, with first and last name 2 Correct
8 Average unit price per category name 2 Correct
9 Top three products by revenue 2 Error: syntax and formula
10 Month of 2016 with the most orders 1 Correct (2016-07)
11 Top five countries by revenue 3 Error: invented join and columns
12 Customers who ordered more in 2016 than in 2015 2 Wrong logic, empty result

Seven of twelve correct. A strict result comparison says five, because questions 2 and 10 returned the right answer in a different shape. On single-table questions it got five of six. On anything with three tables or a comparison across groups, it failed.

The speed is the part that surprised me least and pleased me most:

MLX (mlx-lm 0.32.2, bfloat16) llama.cpp (build 10809, Q8_0)
Model on disk 1,192,135,096 bytes 639,446,880 bytes
Load time (cached) 2.26 s not measured
Generation speed 107.6 to 114.9 tokens/s 125.55 and 131.60 tokens/s (two measured calls)
Time per answer 0.35 to 1.08 s 0.43 s including SQLite
Memory 1.954 GB peak 1,323 MB resident

Both runtimes produced the same SQL for the questions I checked on both.

Where the limits are

The misses are more instructive than the hits. Each one is a pattern you will meet with your own data.

It invents plausible names. In question 5 it grouped by ShipperID. The column is ShipVia; ShipperID only appears in the foreign key clause pointing at another table. A small model pattern-matches on what a column should be called.

It does not quote awkward identifiers. Northwind has a table called Order Details, with a space. The model wrote FROM Order Details, which is a syntax error. When I put a view without the space in front of it (CREATE VIEW OrderDetails AS SELECT ... FROM "Order Details"), the syntax problem went away.

It takes your words literally, and loosely. I asked for revenue as "unit price times quantity, minus the discount". In Northwind, Discount is a fraction between 0 and 1, so the correct expression is UnitPrice * Quantity * (1 - Discount). The model wrote UnitPrice * Quantity - Discount, then with the view UnitPrice * Quantity - Discount * UnitPrice. Only when I wrote the formula out did it produce exactly the right numbers. To be fair, my phrasing was ambiguous. A human analyst would have asked. The model cannot.

Three tables is too many. For the revenue-by-country question it joined Shippers, a table I had not given it, and read the country from columns that do not exist there. Clean table names did not help. This matches the model card: best with one or two tables.

Comparisons across groups break it. "More in 2016 than in 2015" needs conditional aggregation or two subqueries. It produced a self-join comparing freight sums for 2016 against 2016. Valid SQL, meaningless answer, empty result.

Correct SQL can still mislead. Question 6 is my favorite. The model grouped orders by company name and answered "IT", with 335 orders. My reference grouped by customer ID and answered "B's Beverages" with 210. Both queries are reasonable. The difference: this Northwind variant contains two different customers whose company name is "IT". Grouping by name silently merged them. No model, of any size, can know that from the schema. That is a data question, and it is the kind of thing that turns a demo into a wrong decision.

Where the value is

Given all that, would I use it? Yes, in the right seat.

  • As a drafting tool for people who read SQL. It writes the boring 80% of a query in under a second. You check the join and the grouping key.
  • For single-table lookups on local data. Counts, filters, top N, date ranges. This is where it got almost everything right.
  • Inside your own tools, behind a fixed schema. If you control the table names (no spaces, clear column names) and the set of tables per question, you remove most of the failure modes above. Views are a cheap way to give a messy database a clean face.
  • Where data must not leave the machine. Customer data, machine data, anything under a contract that says "not to a third party". A model that runs offline in 1,323 MB of memory changes what is allowed, not just what is possible.

Where I would not use it: anywhere a non-technical user reads the answer without seeing the SQL. Question 6 shows why.

Outlook

Two things make me optimistic about this class of model.

First, the gap is in reasoning across tables, not in SQL itself. That is exactly what targeted fine-tuning and better prompting (schema linking: first pick the tables, then write the query) can address, and both stay cheap at 0.6B parameters. Distil Labs also publishes a 4B variant, which I have not tested here.

Second, the surrounding system matters more than the model. A clean schema, a view layer, a step that shows the SQL before it runs, and a check that the query only reads: those turn a model that gets seven of twelve right into a useful tool. It is the same idea I keep coming back to in my work on fleet data: bring the software to the data, keep the data where it is, and make every answer inspectable.

Frequently asked questions

Can a 0.6B model write correct SQL?

For simple questions, yes. In my test against the Northwind SQLite database, distil-qwen3-0.6b-text2sql answered seven of twelve questions correctly, including five of six single-table questions. It failed on three-table joins, on comparisons across groups, and on a table name containing a space.

Do I need a GPU to run it on a Mac?

No separate GPU is needed. On a MacBook Pro with an M1 Pro and 16 GB of memory, the model loaded in 2.26 seconds with MLX, generated 107.6 to 114.9 tokens per second and used at most 1.954 GB of memory. Answers took between 0.35 and 1.08 seconds.

Why do I need to convert the model for llama.cpp?

The model card mentions an included GGUF file, but as of November 20, 2026 the Hugging Face repository only contains the safetensors weights. MLX loads those directly. llama.cpp and Ollama need GGUF, so you convert it once with llama.cpp's convert_hf_to_gguf.py; the Q8_0 result is 639,446,880 bytes.