Rakesh
Saraswat
Software and AI/ML engineer building production systems end to end: models, services, infrastructure and the products on top.
Software Engineer, San Francisco, CA
Open to full-time roles.
What I work on
I work where models meet production: training and evaluation, the services and infrastructure that serve them, and the products built on top. I like everything I ship to come with the tests and metrics that back it up.
Selected work
PythonFastAPIsentence-transformersStreamlitSQLite
System diagram: Corpus, 34 docs, then BM25, then Dense, then RRF fusion, then Cross-encoder, then Recall@K, MRR, nDCG.
LexSearch EvalBench
A RAG platform built around the evaluation loop, so every claim of better retrieval comes with a number.
- 50-question golden set
- Recall@K, MRR, nDCG
- 33 offline tests
- No API keys
- Hybrid retrieval: BM25 and dense embeddings fused with RRF, then a cross-encoder reranker.
- A labelled golden set of 50 questions over 34 docs, including adversarial sibling documents.
- Reports Recall@K, MRR, nDCG, citation faithfulness, latency and cost, with ablations over chunk size, overlap and fusion weight.
- Runs offline with no API keys and degrades to a hashing embedder and lexical reranker.
PythonFastAPIRedisPostgresPrometheus
System diagram: Client, then Auth, rate limit, PII, then Policy router, then Exact + semantic cache, then Breakers, then Small, then Large, then Premium, then Prometheus.
RouteWise
An OpenAI-compatible gateway that sits in front of several LLM providers and decides where each request goes.
- OpenAI-compatible API
- Exact + semantic cache
- Circuit breakers
- PII redaction
- Policy router picks a model by prompt complexity and downgrades when a budget runs out.
- Exact and semantic caching, circuit breakers, retries and provider fallbacks.
- PII is redacted before anything is logged; SSE streams go through the same pipeline.
- Prometheus metrics and a Streamlit admin dashboard.
SwiftUIReact NativeFastAPIGeminiHybrid RAG
System diagram: Camera, then Vision + OCR, then Manuals, then Hybrid RAG, then Safety gate, then Cited steps, then Refuse, route to a pro.
FixIt Lens
Point a phone at a broken device and get cited repair steps, or a refusal when the repair is dangerous.
- 32 retrieval cases
- 27 safety cases
- 6 OCR cases
- 12 blocked categories
- Vision, retrieval, safety and generation run as separate stages, with Gemini to OpenAI to mock failover.
- Enforced in code: no citation and no safety classification means no repair instruction.
- A native SwiftUI app and an Expo app on a typed FastAPI backend.
- Its own eval suite: 32 retrieval, 27 safety and 6 OCR cases, rerun on every build.
PyTorchTRLvLLMQwen2.5MLflow
System diagram: SFT, 800 trajectories, then Qwen2.5-3B policy, then vLLM rollouts, then FixIt tools, then Verifier reward, then GRPO update.
RepairGym
An RL environment and post-training loop built on FixIt Lens's tools, rewarding answers that cite the right source and pass safety.
- Verifiable reward
- GRPO on one H100
- Reward hack found and patched
- MLflow tracked
- Gym-style environment over the FixIt Lens tools with a verifiable reward: gold chunk cited, safety check passed, a cost for every extra tool call.
- Qwen2.5-3B LoRA SFT on about 800 filtered trajectories, then TRL GRPO on a single H100 with vLLM rollouts.
- Caught the policy reward hacking and closed it: cited text must actually appear in the answer.
- Every run tracked in MLflow.
C++20CMakeGoogle BenchmarkGoogleTestperf
System diagram: ITCH replay, then Parser, then SPSC ring, then Matcher, then Market maker, then Fills, P&L.
MatchBox
A low-latency C++ limit order book and matching engine that replays Nasdaq ITCH, with a simple market-making strategy on top.
- Over 1M msg/s
- Median add/cancel under 500 ns
- Lock-free SPSC ring
- GoogleTest in CI
- Allocation-free hot path with pool-allocated orders and intrusive per-level lists: over 1M messages a second, median add/cancel under 500 ns.
- Pinned feed and matcher threads joined by a lock-free single-producer, single-consumer ring.
- Profiled with perf and Google Benchmark to remove false sharing and branchy lookups; GoogleTest in GitHub Actions.
- The strategy quotes both sides with inventory skew and tracks fills and P&L.
GoRustKubernetesTemporalNATSvLLM
System diagram: CRDs, then Operator, then Bin-packing, then Temporal lifecycle, then NATS JetStream.
RackLoop
A Kubernetes-native control plane for a small GPU inference fleet: declare models and clusters, and it provisions hosts, heals faults and packs GPUs.
- 256 KWOK nodes replayed
- Temporal state machine
- Redfish remediation
- envtest in CI
- Go operator (kubebuilder) with InferenceCluster and ModelDeployment CRDs; vLLM and SGLang behind one interface.
- Host lifecycle (discovery, iPXE imaging, driver install, burn-in, drain, RMA) as a versioned Temporal state machine.
- A Rust node agent publishes GPU faults (XID, ECC, NCCL all-reduce checks) to NATS JetStream; remediation drains, power-cycles over Redfish and re-admits.
- Best-fit bin-packing and defragmentation replayed on 256 KWOK nodes against a public GPU cluster trace.
More projects
8 in total- AnswerRankA ranking-model training and serving platform: a DLRM-style PyTorch ranker served from Triton, a Go ranking service over gRPC, Feast features and a parity-checked canary deploy.PyTorch, Ray, Triton, 2026
- DiffWiseA VS Code and Cursor extension plus GitHub App that turns AI-written pull requests into a review guide: hunks grouped by function, behaviour changes first, untested code flagged.TypeScript, tree-sitter, Probot, 2026
- TokenScopeLatency forensics for LLaMA-family inference: where every millisecond of a generated token goes.Python, PyTorch, llama.cpp, 2026
- WorkGraphA work knowledge graph across Gmail, Drive, Slack, GitHub and Linear, with workflow agents where every integration is an MCP server and actions wait for human approval.FastAPI, Postgres, pgvector, 2026
How I like to work.
I'm a software engineer based in San Francisco, working across the whole stack and into the model: native iOS and React Native apps, React and Next.js frontends, Python and Go services, C++ where latency matters, and ML from retrieval and evaluation to post-training.
Whatever the layer, I aim for the same standard: well tested, measured, and clear about how it behaves when something goes wrong.
Experience and education
Software Developer, California State University, Long Beach
Aug 2025 – Sep 2026
Owned the Python sync services for graduate application review and deployed an XGBoost classifier that flags missing materials.
Software Engineer, PayGuru TechnoSoft
Dec 2023 – Dec 2024
Built Electron and C++ software for attendance and access control, and real-time management for 1,000+ devices.
M.S. Computer Science, California State University, Long Beach
Expected Dec 2026
GPA 4.0 / 4.0
I measure what I build.
EvalBench scores retrieval on a labelled golden set with Recall@K, MRR and nDCG. TokenScope puts confidence intervals on latency.
I build for graceful failure.
RouteWise trips circuit breakers and falls back across providers. FixIt Lens drops from Gemini to OpenAI to a deterministic mock.
I ship complete products.
FixIt Lens ships as a native SwiftUI app and an Expo app, not just an API. DiffWise puts its review guide inside the editor where reviewers already work.
What I build with
Languages
- Python
- Java
- C++
- TypeScript
- JavaScript
- SQL
- Swift
Backend and data
- FastAPI
- Node.js
- REST and WebSockets
- SQLAlchemy
- PostgreSQL
- Redis
- MongoDB
Machine learning
- PyTorch
- scikit-learn
- XGBoost
- pandas
- MLflow
- Computer vision
LLM systems
- RAG and reranking
- Embeddings
- Semantic caching
- LLM evaluation
- OpenAI and Gemini APIs
- llama.cpp
Infrastructure
- Docker
- GCP Cloud Run and Cloud SQL
- GitHub Actions CI/CD
- Prometheus
- pytest
- Linux
Frontend and apps
- React
- Next.js
- React Native
- SwiftUI
- Electron
- Three.js and GLSL
Hiring for a full-time role? I’d love to hear from you.
Thank you for stopping by. I read every message and usually reply within a couple of days, and I’m open to software engineering, full-stack and AI/ML roles.