How to Run a Local AI SQL Assistant with Ollama

# How to Run a Local AI SQL Assistant with Ollama

TL;DR -- Key Takeaways

>

- Ollama lets you run large language models entirely on your Mac, turning your laptop into a private AI SQL generator.

- Local AI SQL means zero API costs, zero data exfiltration, and full GDPR/SOC 2 compliance by default.

- Models like CodeLlama, DeepSeek-Coder, SQLCoder, and Mistral can generate production-grade SQL from plain English.

- QueryDeck detects a running Ollama instance automatically and uses it as a backend with no configuration required.

- The tradeoff: local models are slower and slightly less accurate than cloud models, but for most day-to-day SQL work the difference is negligible.

Every time you paste a schema into ChatGPT or send a prompt to a cloud API, your table names, column names, and sometimes your actual data leave your machine. For side projects, that might be fine. For production databases holding customer records, medical data, or financial transactions, it is a compliance risk you do not need to take.

An ollama sql assistant running on your own hardware eliminates that risk entirely. No network requests, no third-party data processing agreements, no API bills that scale with usage. Just a model, your schema, and the SQL you need.

This tutorial walks you through the full setup: installing Ollama on macOS, choosing the right model for SQL generation, connecting it to a database client, and understanding when local AI is the right call versus when you should reach for a cloud model.

Why Run AI Locally for SQL?

Before diving into the how, it is worth being explicit about the why. There are four reasons developers and database administrators choose a local AI SQL setup over cloud-based alternatives.

Privacy and compliance

When you use a cloud-based AI SQL generator, your prompts -- including schema details and sometimes query results -- travel over the internet to a third-party server. Even if the provider promises not to train on your data, you are still transmitting it. Under GDPR, HIPAA, SOC 2, and many internal security policies, that transmission itself can be a violation.

A private AI SQL generator running through Ollama keeps everything on your machine. The model weights live on your disk. Inference happens on your CPU or GPU. Nothing leaves localhost.

No API costs

Cloud AI APIs charge per token. If you are generating SQL dozens of times a day -- exploring schemas, writing migrations, building reports -- those tokens add up. With Ollama, inference is free after the initial model download. You are paying in electricity and compute time, not per request.

Offline capability

Cloud APIs require an internet connection. If you are working on a plane, in a coffee shop with unreliable Wi-Fi, or in a secure facility without internet access, a cloud-based AI assistant is useless. An offline AI SQL assistant works anywhere your laptop does.

Self-hosted control

With a self-hosted SQL AI setup, you control the model version, the context window, and the update cycle. No surprise model changes, no deprecated endpoints, no rate limits. You decide when to pull a new model and when to stick with what works.

How to Install Ollama on Mac

Ollama is the easiest way to run open-source LLMs locally. The installation takes under five minutes.

Step 1: Download Ollama

Visit ollama.com and download the macOS installer. Alternatively, if you use Homebrew:

brew install ollama

Step 2: Start the Ollama server

After installation, launch the Ollama application from your Applications folder. It runs as a menu bar icon and starts a local API server on http://localhost:11434.

You can verify it is running:

curl http://localhost:11434/api/tags

This should return a JSON response listing your installed models (empty at first).

Step 3: Pull your first model

Ollama models are pulled on demand. To download a model suitable for SQL generation:

ollama pull codellama:13b

The download size varies by model. CodeLlama 13B is roughly 7 GB. Once downloaded, the model stays on disk and loads into memory when you run it.

Step 4: Test it

Run a quick interactive test:

ollama run codellama:13b "Write a SQL query to find all users who signed up in the last 30 days"

If you get a valid SQL response, your ollama sql assistant is operational.

Best Models for SQL Generation

Not all LLMs are equal at writing SQL. Here are the models that consistently perform well for ollama database tasks, ranked by SQL-specific capability.

SQLCoder (defog/sqlcoder)

SQLCoder is purpose-built for text-to-SQL conversion. It was fine-tuned by Defog on SQL-specific datasets and outperforms general-purpose models on schema-aware SQL generation.

ollama pull defog/sqlcoder

Best for: Production SQL generation where accuracy matters most. Handles complex JOINs, subqueries, and window functions reliably.

Tradeoff: Narrowly specialized. Less useful for general code explanation or non-SQL tasks.

DeepSeek-Coder

DeepSeek-Coder is a strong general coding model with excellent SQL capabilities. The 6.7B and 33B variants both handle SQL well.

ollama pull deepseek-coder:6.7b

Best for: Mixed workflows where you write SQL alongside application code. Good at explaining existing queries and suggesting optimizations.

Tradeoff: Not as specialized as SQLCoder for pure text-to-SQL, but more versatile.

CodeLlama

Meta's CodeLlama is the most widely used coding model in the Ollama ecosystem. The 13B parameter version hits a sweet spot between quality and speed on Apple Silicon Macs.

ollama pull codellama:13b

Best for: General SQL generation, query explanation, and schema exploration. Large community and extensive documentation.

Tradeoff: The 7B version can struggle with complex multi-table JOINs. Use 13B or higher for production work.

Mistral

Mistral 7B punches above its weight class. It is not a coding-specific model, but its general reasoning ability translates well to SQL tasks, especially for simpler queries and data exploration.

ollama pull mistral

Best for: Lightweight SQL generation on machines with limited RAM (runs well with 8 GB). Good at natural language explanations of query results.

Tradeoff: Less reliable on complex SQL syntax compared to the coding-focused models above.

Model comparison at a glance

How to Connect Ollama to a Database Client

Running Ollama in a terminal is useful for testing, but the real productivity gain comes from integrating it with a database client that understands your schema.

The manual approach: API + scripting

Ollama exposes a REST API at localhost:11434. You can script natural language to SQL by sending your schema as context:

curl http://localhost:11434/api/generate -d '{
  "model": "codellama:13b",
  "prompt": "Given the following PostgreSQL schema:\n\nCREATE TABLE orders (id serial PRIMARY KEY, user_id int, total decimal, created_at timestamp);\nCREATE TABLE users (id serial PRIMARY KEY, email text, name text);\n\nWrite a query to find the top 10 customers by total order value.",
  "stream": false
}'

This works, but manually copying your schema into every prompt is tedious and error-prone.

The integrated approach: QueryDeck + Ollama

QueryDeck is a database client built for app developers that supports Ollama as an AI backend out of the box. With ORM auto-detection and SQL notebooks built in, the integration is designed to be zero-configuration:

1. Install Ollama and pull a model (as described above).
2. Open QueryDeck and connect to your PostgreSQL, MySQL, or SQLite database.
3. QueryDeck detects Ollama automatically. No API keys to paste, no endpoints to configure. If Ollama is running on localhost:11434, QueryDeck finds it.
4. Ask questions in plain English using the AI assistant panel. QueryDeck sends only your schema (table names and column names) to the local model -- never your actual data.

The result is a private AI SQL generator that combines the convenience of cloud-based AI assistants with the privacy of local inference.

What makes this setup particularly compelling:

• Schema-only context: QueryDeck reads your database schema and sends table/column metadata to Ollama. Your row data never touches the model.
• Works completely offline: Disconnect your Wi-Fi. The AI assistant keeps working because both QueryDeck and Ollama run locally.
• Same UX as cloud models: The prompt interface, SQL preview, and query execution flow are identical whether you are using Ollama, OpenAI, or Anthropic as the backend. You can switch between local and cloud models without changing your workflow.

Practical Examples: Natural Language to SQL

Here are real-world examples of what a local AI SQL assistant can generate. These were tested with CodeLlama 13B running on an M3 Pro MacBook Pro.

Example 1: Simple aggregation

Prompt: "How many orders were placed each month this year?"

Generated SQL:

SELECT
  DATE_TRUNC('month', created_at) AS month,
  COUNT(*) AS order_count
FROM orders
WHERE created_at >= DATE_TRUNC('year', CURRENT_DATE)
GROUP BY DATE_TRUNC('month', created_at)
ORDER BY month;

Accuracy: correct on first attempt. The model correctly used DATE_TRUNC for PostgreSQL rather than MySQL's DATE_FORMAT.

Example 2: Multi-table JOIN with filtering

Prompt: "Show me all users who have never placed an order."

Generated SQL:

SELECT u.id, u.email, u.name
FROM users u
LEFT JOIN orders o ON u.id = o.user_id
WHERE o.id IS NULL;

Accuracy: correct. The model chose a LEFT JOIN with NULL check, which is idiomatic and efficient.

Example 3: Window function

Prompt: "Rank products by revenue within each category."

Generated SQL:

SELECT
  p.name AS product_name,
  c.name AS category_name,
  SUM(oi.quantity * oi.unit_price) AS revenue,
  RANK() OVER (PARTITION BY p.category_id ORDER BY SUM(oi.quantity * oi.unit_price) DESC) AS rank
FROM products p
JOIN categories c ON p.category_id = c.id
JOIN order_items oi ON p.id = oi.product_id
GROUP BY p.id, p.name, c.name, p.category_id
ORDER BY c.name, rank;

Accuracy: correct structure. The model inferred reasonable table relationships from column naming conventions.

For a deeper dive into natural language SQL generation, see our guide on natural language to SQL in 2026.

Performance Comparison: Local vs Cloud

The honest answer is that cloud models are still ahead on raw capability. But "ahead" does not always mean "necessary." Here is how the two approaches compare in practice.

Speed

Local inference is slower per query, but the absence of network latency and rate limits means you can iterate faster in practice. You never wait for an API timeout or hit a "too many requests" error.

Accuracy

For straightforward SQL -- SELECTs, JOINs, GROUP BY, common aggregations -- local models running 13B+ parameters match cloud models in most cases. The gap appears with:

• Highly complex CTEs with multiple recursive levels
• Database-specific syntax for less common engines
• Ambiguous prompts that require stronger reasoning to disambiguate
• Very large schemas (100+ tables) where context window limits matter

For the 80% of SQL work that involves standard queries against well-structured schemas, a local model is good enough.

Cost

The only local cost is the electricity to run inference, which is negligible on Apple Silicon.

When Local Is Good Enough vs When You Need Cloud

Use local (Ollama) when:

• You work with production databases containing sensitive data
• Your company has strict data residency or compliance requirements (GDPR, HIPAA, SOC 2)
• You are generating routine SQL: CRUD operations, reports, data exploration
• You want to avoid recurring API costs
• You need to work offline or in air-gapped environments
• You are exploring your own schema and want fast iteration without rate limits

Use cloud models when:

• You are working with extremely complex analytical queries (nested CTEs, advanced window functions, recursive queries)
• You need to generate SQL for unfamiliar or niche database engines
• You want the highest possible accuracy and are willing to pay for it
• Your schema is very large (100+ tables) and exceeds local model context windows
• You are building SQL for data pipelines where correctness is critical and will not be manually reviewed

The hybrid approach

The best setup for most developers is both. Use a local Ollama model for daily SQL work -- the queries you write dozens of times a day where privacy matters and "good enough" is truly good enough. Switch to a cloud model when you hit a query that stumps the local model or when you need maximum accuracy for a one-time migration script.

QueryDeck supports this hybrid workflow natively. You can configure both Ollama and a cloud provider (OpenAI, Anthropic) and switch between them per query. The interface stays the same; only the backend changes.

Security and Privacy Benefits for Production Databases

Using a self-hosted SQL AI is not just a nice-to-have for security-conscious teams. It is increasingly a requirement.

What stays on your machine

When you use Ollama with a schema-aware client like QueryDeck:

• Model weights: Downloaded once, stored on your local disk
• Schema metadata: Table names and column names are sent to the local model as context -- they never leave your machine
• Row data: Never sent to the model. The AI generates SQL; you execute it yourself against your database
• Query history: Stored locally, not in a third-party's logs

What this means for compliance

• GDPR Article 28: No data processor agreement needed because no data is processed by a third party
• SOC 2: No third-party subprocessor to evaluate, no data flow diagrams to update
• HIPAA: No Business Associate Agreement required for the AI component
• Internal security policies: No need for security review of an external AI vendor

Air-gapped and restricted networks

Some environments -- government agencies, financial institutions, healthcare organizations -- operate on networks with no internet access. A local AI SQL assistant is the only option in these settings. Install Ollama and pull your model while connected, then work indefinitely without a network.

Getting Started in Five Minutes

Here is the fastest path from zero to a working ollama sql assistant:

# 1. Install Ollama
brew install ollama

# 2. Start the server
ollama serve &

# 3. Pull a SQL-capable model
ollama pull codellama:13b

# 4. Verify it works
ollama run codellama:13b "Write a PostgreSQL query to list all tables in the current database"

Then connect it to your workflow:

• For a full GUI experience: Download QueryDeck, connect your database, and start asking questions. Ollama is detected automatically.
• For terminal-based workflows: Use the Ollama API directly or pipe prompts through a script that includes your schema.
• For VS Code users: Several extensions support Ollama as a backend for inline SQL generation.

For more on choosing the right database client to pair with Ollama, see our comparison of the best database clients for Mac in 2026.

Conclusion

Running a local AI SQL assistant with Ollama is no longer an experiment -- it is a practical, production-ready workflow. The models are capable enough for daily SQL work, the tooling has matured, and the privacy benefits are significant for anyone working with real data.

The setup takes five minutes. The API cost savings start immediately. And your schema never leaves your machine.

If you are already using a cloud-based AI for SQL generation, try running the same prompts through Ollama locally. For the vast majority of queries, you will not notice a difference in output quality. But you will notice the difference in your API bill and your compliance posture.

Start with CodeLlama 13B for a balance of speed and accuracy, or SQLCoder if pure text-to-SQL precision is your priority. Pair it with QueryDeck for the smoothest integration, or use the Ollama API directly if you prefer scripting. Either way, your data stays yours.