I build ML systems — and then I try to break them with measurement.
Most AI demos work. The interesting question is how they fail, how often, and what that costs. That's the question I keep building around.
I spend most of my time on evaluation harnesses for LLM systems: small, self-contained studies that take a plausible-sounding design claim and put a number on it.
The pattern is always the same — pick a system someone is shipping, build the benchmark it doesn't have, and report what actually happens, including when the answer is "this doesn't work."
A model that looks right 90% of the time is a product decision, not a result. The job is knowing which 10%, and what it costs when it happens.
Before this, my foundation was Graph Neural Networks and financial fraud research — the same rigor, applied to shipped systems instead of benchmarks.
Each repo is a measured claim, not a demo.
| Project | The Question | The Finding |
|---|---|---|
| cascade | Does cheap-model-first LLM routing actually save money? | 98.9% of strong-model accuracy at 69.1% of cost — but 57% of the gain is self-consistency, not routing, and it's 2.4× slower. You're trading latency for money. |
| wisdom-sql | Is that feedback signal learning, or is it an oracle? | The "oracle" signal is a perfect selector (32/32) on text-to-SQL. 67% of the reported lift disappears once it can't see the gold answer. |
| slide-faithful | How often does a "board-ready" AI deck invent a number? | A board-ready + impactful prompt fabricates a figure on 60% of slides — 100% on growth rates and projections. |
| formi-outcomes | What if the LLM judging "call succeeded" is also the billing meter? | A naive outcome-judge over-bills 30% of calls; 55% of what it bills is a phantom collection. |
| tf-routing | Is cheap-first routing viable under a real SLA? | 56% cheaper — with a p99 at 2× the strong model's. Infeasible under the SLA it was proposed for. |
| catalog-gap | Can a shopping agent answer what the catalog never said? | 1,198 live D2C products: 78% declare grams: 0, and the agent repeats it 96% of the time — fully grounded, entirely wrong. |
| homi-stepmarks | Can a cheap grader mark the method, not just the answer? | A step-level marker for CBSE algebra — the cheap grader calls 19 of 40 broken solutions correct. |
| escalation-boundary | Does a medical-practice ops agent know when to stop? | A 60-case escalation benchmark separating the buried red flag from the plain clinical question. |
| matched | Three-way matching (PO / Invoice / GR) as entity resolution | 72.73 F1 with 7 interpretable features, beating DeepMatcher (62.80) — plus an honest failure map: 8.7% wrong-payment rate on hard negatives. |
| Project | Focus | Notes |
|---|---|---|
| TRDGNN | 🔴 Flagship: Temporal GNNs | Time-relaxed directed GNN for Bitcoin fraud detection — multiple architectural contributions, publication-ready analysis |
| GraphTabular-FraudFusion | 📉 Negative result | A rigorous study of when graph embeddings do not improve XGBoost |
| Revisiting-GNNs-FraudDetection | 📊 Reproducibility | GraphSAGE / GCN / GAT vs. XGBoost baselines, honestly compared |
| Research-Paper-Analyzer | 🧠 LLM product | PDF → structured JSON with grounding, numeric consistency, and latency constraints |
- Evaluation harnesses and benchmarks for LLM systems
- LLM routing, cascades, cost/latency tradeoffs under real SLAs
- Hallucination and grounding failure analysis
- Graph-based learning (fraud, anomalies, structured data)
- Entity resolution and interpretable baselines that beat deep models
PyTorch Geometric (GNNs) · evaluation pipelines · ablation studies · failure analysis
Research teaches why. Engineering decides what survives reality. Measurement is how you tell the difference.
I optimize for clarity, correctness, and long-term usefulness — not hype.
📫 Connect
- GitHub: https://github.com/BhaveshBytess
- LinkedIn: https://www.linkedin.com/in/bhavesh-ai