- Designed end-to-end test scenarios for AI coding agents — each packaged as a reproducible Dockerised environment with a written specification, a reference implementation, and an automated verification suite defining objective pass/fail criteria.
- Applied boundary, negative and exploratory testing to surface agent failure modes in long-horizon planning, environment state handling, and recovery from failed tool calls.
- Enforced determinism and eliminated flaky verification so results measure agent capability rather than environment defects.
- Extended five years of test-design discipline to non-deterministic systems, where correctness is probabilistic and assertions must tolerate valid variation in approach while still catching wrong outcomes.
SDET → AI eval · Solo builderRemote · Worldwide
Writing code got cheap. Knowing it is right did not. I work the second half.
Software quality engineer by training — automation, load, and release readiness at scale. Now I evaluate AI coding agents for a living, and I build the kind of system I would otherwise be asked to test: agentic RAG, multi-agent research, and a UPI-native payments core proved out with property-based tests.
Open to select engagements — remote, worldwide. Replies within a day.
5 yrs
Owning quality
across QA, SDET & AI eval roles
across QA, SDET & AI eval roles
0
Critical production defects
across feature releases
across feature releases
60–70%
Manual test effort removed
through automation
through automation
Selected workgithub.com/raghavg27 ↗
§ Current assignment
I test the AI coding agents other people are about to trust.
Correctness used to be binary: run it twice, get the same answer. AI coding agents broke that assumption, and I moved toward the break rather than away from it. I design the evaluation environments that decide whether an agent actually did the job — and the hard part is not the agent, it is making a red run mean exactly one thing.
How I test an agent →- 01SpecificationThe task in prose, the way a ticket would arrive.
- 02EnvironmentDockerised, pinned, no network at solve time.
- 03ReferenceA known-good solution — the control on the verifier.
- 04VerifierObjective pass/fail on outcomes, never on the transcript.
§ Selected work · 4 projects
What I've built and how.
Shipped · open source
Squared Up
UPI-native expense splitting
An India-first Splitwise alternative: split the bill, settle in one UPI tap. Django/DRF over PostgreSQL with a framework-free money core held in integer paise, plus an installable React PWA. The correctness layer is proved against spec vectors and Hypothesis property tests before it ever touches a database.
- Stack
- Django · DRF · PostgreSQL · React PWA
- Money
- Integer paise · no floats
- Proved with
- Spec vectors + Hypothesis
Shipped · open source
Equity Crew
Multi-agent equity research
Five CrewAI agents research a listed stock the way a desk would: fundamentals across eight yfinance endpoints, neural news search, technical indicators written from scratch in pandas, and a dynamically chosen peer set — fused into a BUY/HOLD/SELL call with a confidence score and a generated PDF report.
- Stack
- Python · CrewAI · OpenRouter · EXA
- Agents
- 6 defined · 4 run in parallel
- Output
- Validated schema + PDF report
Shipped · open source
Git Guide
Agentic RAG over GitLab docs
A multi-agent RAG assistant that answers GitLab questions straight from the official documentation — grounded, cited, and thrown away when the retrieved chunks are not good enough to support an answer. CrewAI agents, local ChromaDB retrieval, async smart routing, Streamlit UI, and it runs entirely on free tiers.
- Stack
- Python · CrewAI · ChromaDB · Streamlit
- Embeddings
- BAAI/bge-small · local CPU
- Cost
- $0 to operate
Shipped · on sale
Steno
Writing standard for AI prose
A writing standard that makes AI-generated prose read as though a person wrote it — used as a checklist while drafting, or as an editing pass over text that already sounds machine-written. Rules are tiered so structure and stance get fixed before cosmetic tells. Product, copy, brand and front end built end to end by one person.
- Ships as
- Claude skill · portable prompt
- Model
- One-time purchase
- Built
- Product · copy · brand · front end
§ What I believe
Quality is a habit, not a phase at the end.
Testing taught me the cheapest bug is the one you design out before it exists. The same instinct now runs my products: ship small, watch it break, fix the loop — then add scope. Taste and tight feedback beat headcount.
Principle 01
Break it before your users do.
Every feature earns a way to fail loudly in staging first. Confidence at release is built, never assumed.
Principle 02
Automate the boring, own the judgement.
Machines run the regression pass. I keep the calls that need context — what to test, what to ship, what to hold.
Principle 03
Grounded beats clever.
Whether it's a test suite or a RAG agent, I'd rather ship something cited, predictable, and honest than a demo that hallucinates.
§ The record
Where I've shipped quality.
- Owned end-to-end sprint QA — planning, test design, automation, defect tracking and retrospectives — landing on-time releases with improved defect containment.
- Built automated regression suites for warehouse and robot workflows in PyTest, Selenium, Playwright and Page Object Model, cutting manual effort 60–70%.
- Ran performance and load testing for high-throughput bulk operations with JMeter and REST API testing, validating SLA compliance under peak load.
- Published pipeline metrics to Grafana for coverage, failure-trend and release-readiness visibility across stakeholders.
- Triaged nightly automated pipelines — isolating flaky tests and coordinating fixes via JIRA — reducing false positives 25%.
- Led functional, system and regression testing across iOS and Android using manual QA plus Selenium/Appium — zero critical production defects across releases.
- Ran end-to-end API testing with Postman, RestAssured, Requests and Swagger, cutting backend defect leakage 40%.
- Executed load testing for high-traffic flows with JMeter, improving response times under peak load 30%.
- Designed an in-house data-driven automation framework (PyTest, Page Object Model) that reduced test maintenance 35%.
- Integrated test execution into CI/CD with Jenkins, Git, Docker and Kubernetes for quality checks on every build.
- Acted as quality gatekeeper for enterprise clients — owning pre-release validation, defect triage and go-live sign-offs for conversational workflows.
- Triaged incidents, set severity, and verified fixes in staging and production, lifting CSAT and reducing repeat incidents.
- Led technical UAT sessions with enterprise customers, converting business feedback into product improvements.
§ Toolkit
What I work with.
Full-stack on quality and delivery — from writing the framework to reading the Grafana board at 2 a.m. Comfortable across languages, test tooling, AI/RAG systems, and the DevOps chain that ships it.
SRM Institute of Science and Technology
B.E. Computer Engineering · 2017–2021
Languages & Data
- Python
- JavaScript
- SQL
- PostgreSQL
- MongoDB / NoSQL
- Bash
Test & Automation
- Selenium
- Playwright
- Appium
- PyTest
- Hypothesis
- JUnit
- Page Object Model
- JMeter
- Postman / RestAssured
AI / Backend
- CrewAI · Agentic RAG
- ChromaDB
- Django / DRF
- React
- Node.js
- REST / OpenAPI
DevOps & CI/CD
- Jenkins
- Docker
- Kubernetes
- Git / GitHub Actions
- Grafana
- JIRA · TestRail · Confluence