|
1 | | -# NL2SQL |
| 1 | +# SQL Copilot — Interactive README |
2 | 2 |
|
3 | | -Natural language → SQL platform. Upload a CSV or connect Postgres/MySQL, ask questions in plain English, get validated SQL + results. |
| 3 | +[]() |
| 4 | +[]() |
| 5 | +[]() |
4 | 6 |
|
5 | | -## Flow |
6 | | -Frontend (NL question) → embed question → RAG retrieves relevant tables from schema (Chroma) → FastAPI builds prompt with schema + dialect → Groq LLM generates SQL → sqlglot validates (blocks non-SELECT, checks tables exist, enforces row limit) → executes against DB → self-correction retry on failure → plain-language answer. |
| 7 | +SQL Copilot translates natural-language questions into SQL and helps you run, test, and iterate on queries safely. This README includes a small interactive demo you can run locally to try NL→SQL conversion and execute queries on a sample dataset. |
7 | 8 |
|
8 | | -## Setup |
| 9 | +Quick highlights |
| 10 | +- Interactive local demo (Streamlit) to try NL→SQL conversion and query an example SQLite DB. |
| 11 | +- Support for plugging in your LLM (OpenAI, other APIs) or using a simple fallback translator for offline demos. |
| 12 | +- Safety notes and recommended .gitignore entries to avoid committing secrets or DB dumps. |
9 | 13 |
|
10 | | -```bash |
11 | | -cd backend |
12 | | -python3 -m venv venv |
13 | | -source venv/bin/activate |
14 | | -pip install -r requirements.txt --break-system-packages |
| 14 | +Live / Interactive demo (local) |
| 15 | +1. Install dependencies |
| 16 | + pip install -r requirements.txt |
| 17 | + pip install streamlit |
15 | 18 |
|
16 | | -export GROQ_API_KEY="your_key_here" |
17 | | -uvicorn main:app --reload --port 8000 |
18 | | -``` |
| 19 | + Optional (LLM mode): |
| 20 | + pip install openai python-dotenv |
19 | 21 |
|
20 | | -Open `frontend/index.html` in a browser (or serve it). |
| 22 | +2. Copy environment example and add API key if you want to use an LLM: |
| 23 | + cp .env.example .env |
| 24 | + # set OPENAI_API_KEY or other provider key in .env |
21 | 25 |
|
22 | | -## Endpoints |
23 | | -- `POST /upload` — upload CSV, builds SQLite dataset + schema index |
24 | | -- `POST /connect` — connect existing Postgres/MySQL DB via SQLAlchemy conn string |
25 | | -- `POST /query` — ask a question, returns `{sql, columns, rows, answer}` |
| 26 | +3. Run the playground: |
| 27 | + streamlit run interactive/playground.py |
26 | 28 |
|
27 | | -## Notes |
28 | | -- Only SELECT statements are ever executed (sql_guard.py blocks DROP/DELETE/UPDATE/etc.) |
29 | | -- RAG (Chroma + bge-small embeddings) only kicks in past 6 tables — small schemas get full context, no retrieval overhead |
30 | | -- One LLM self-correction retry on execution error |
31 | | -- Read-only DB user recommended in production for the Postgres/MySQL connection string |
| 29 | +4. In the demo: |
| 30 | + - Enter a natural-language request (e.g. "List top 5 customers by revenue in 2024"). |
| 31 | + - Click "Translate" to show the generated SQL. |
| 32 | + - Click "Run SQL" to execute it against an in-memory sample SQLite DB and view results. |
| 33 | + |
| 34 | +Why this README is "interactive" |
| 35 | +- It ships a lightweight Streamlit playground so reviewers and maintainers can try NL→SQL without provisioning a DB or keys. |
| 36 | +- The playground supports two modes: |
| 37 | + - LLM mode (requires API key) — demonstrates production-style usage. |
| 38 | + - Fallback mode (no key) — shows a deterministic / heuristic conversion so the demo always runs offline. |
| 39 | + |
| 40 | +Quickstart (CLI) |
| 41 | +- Translate and print SQL (example CLI wrapper): |
| 42 | + python -m sql_copilot.cli --nl "Give me the top 10 orders by amount" |
| 43 | + |
| 44 | +- Run the local Streamlit demo: |
| 45 | + streamlit run interactive/playground.py |
| 46 | + |
| 47 | +Playground (what the demo does) |
| 48 | +- Builds a small example SQLite DB (tables: customers, orders, products). |
| 49 | +- Shows schema and sample rows. |
| 50 | +- Accepts a NL question and: |
| 51 | + - If OPENAI_API_KEY (or other provider env var) is present, calls the provider to convert NL→SQL using a safe prompt template that includes schema and execution constraints. |
| 52 | + - Otherwise uses a conservative heuristic fallback to create a simple SELECT query or asks the user to refine the request. |
| 53 | +- Executes SQL and displays results in a table with an execution trace. |
| 54 | + |
| 55 | +Security / Safety |
| 56 | +- Never commit real DB dumps, credentials, or production data to this repo. |
| 57 | +- Add these to `.gitignore`: |
| 58 | + - .env |
| 59 | + - *.db |
| 60 | + - data/ |
| 61 | + - dbs/ |
| 62 | + - models/ |
| 63 | + - checkpoints/ |
| 64 | + - logs/ |
| 65 | +- When using external LLMs, sanitize or avoid sending sensitive data to third-party APIs. Prefer local models or private endpoints for regulated data. |
| 66 | + |
| 67 | +Contributing |
| 68 | +- Please add tests for any new translator logic. |
| 69 | +- Keep example/data generation deterministic (seeded random) so demos are reproducible. |
| 70 | +- If adding new provider integrations, add an example provider config to `docs/providers.md` (do not include secrets). |
| 71 | + |
| 72 | +Troubleshooting |
| 73 | +- If you see errors in playground: |
| 74 | + - Ensure Python >= 3.8 is used. |
| 75 | + - If using LLM mode, confirm OPENAI_API_KEY is set and you have network access. |
| 76 | + - If streamlit is slow, reduce the sample DB size or run with fewer UI widgets. |
| 77 | + |
| 78 | +License & Contact |
| 79 | +- MIT License (or choose appropriate license). |
| 80 | +- Maintainer: @10div10 — open issues or PRs for improvements. |
0 commit comments