RAG Study Assistant (FastAPI + ChromaDB + Transformers)

A compact Retrieval-Augmented Generation (RAG) assistant for studying course PDFs/notes.

• Ingest .pdf / .txt / .md
• Persisted vector store (ChromaDB)
• Local embeddings (Sentence-Transformers)
• Local generator (FLAN-T5) with safe fallback
• Simple evaluation script and Dockerfile

Quickstart

# 1) Create env
python -m venv .venv && . .venv/bin/activate  # on Windows: .venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt

# 2) Ingest sample docs (or place your own files under data/samples)
python scripts/ingest.py --data data/samples --persist .chromadb

# 3) Run API
uvicorn src.app.main:app --host 0.0.0.0 --port 8000 --reload
# Open http://127.0.0.1:8000/docs for a swagger UI

# 4) Ask questions
curl -X POST http://127.0.0.1:8000/query -H "Content-Type: application/json" -d '{"question": "What is Big-O for binary search?"}'

Project Layout

.
├─ README.md
├─ requirements.txt
├─ .env.example
├─ data/
│  ├─ samples/                 # put your PDFs/notes here
│  └─ eval/questions.jsonl     # tiny demo eval set
├─ scripts/
│  ├─ ingest.py                # ingest PDFs/txt/md into Chroma
│  └─ evaluate.py              # minimal evaluation (retrieval + generation quality)
├─ src/
│  ├─ app/main.py              # FastAPI app
│  ├─ app/schemas.py
│  ├─ retriever/index.py
│  ├─ models/generator.py
│  └─ eval/metrics.py
├─ tests/
│  └─ test_smoke.py
└─ Dockerfile

Environment variables

Copy .env.example to .env and tweak:

• CHROMA_DIR: where to persist chroma index (default: .chromadb)
• EMBEDDING_MODEL: sentence-transformers model name (default: all-MiniLM-L6-v2)
• LLM_MODEL_NAME: text2text model for generation (default: google/flan-t5-base)
• MAX_CONTEXT_CHARS: cap total retrieved context length (default: 3000)

Notes

• First run will download models (embeddings + generator).
• If FLAN-T5 is too heavy, change LLM_MODEL_NAME to google/flan-t5-small.
• If you prefer OpenAI as the generator, see comments in src/models/generator.py (off by default).

Docker (optional)

docker build -t rag-study .
docker run --rm -it -p 8000:8000   -v $PWD/.chromadb:/app/.chromadb   -v $PWD/data:/app/data   rag-study

Evaluation

python scripts/evaluate.py --questions data/eval/questions.jsonl --k 4

This prints:

• Retrieval hit rate@k
• A rough ROUGE-L against references (toy metric for demo)