How I Built RagXray: A $0 Serverless RAG Playground with Hybrid Search, RRF and Real-Time Self-Evaluation

I wanted to understand how RAG actually works under the hood. Not the diagram version, the real version.

The problem is that most RAG pipelines are a black box. You write a prompt, query a vector database, hit an LLM, and get an answer back. If the answer is bad, you have no idea which stage failed. Was the chunking wrong? Did retrieval fetch noise? Did the model hallucinate over good context?

So I built RagXray, an interactive playground that visualizes every single stage of a RAG pipeline in real time. You can toggle each step on and off, vector search, BM25, RRF, re-ranking, and watch how it changes both latency and answer quality.

You can try it here: ragxray-web.pages.dev

This post walks through how it works and the decisions behind it. The whole thing runs serverless and costs $0 a month, which forced some interesting engineering along the way.

The stack

Next.js 15 for the frontend, Hono for the API layer, Cloudflare Workers AI for embeddings, Neon Postgres with pgvector for vector search, and Cloudflare D1 (SQLite at the edge) for keyword search. Gemini 2.5 Flash handles generation and evaluation.

No LangChain, no Pinecone, no managed search cluster. Part of the point was proving you can build a serious hybrid RAG pipeline out of cheap primitives you actually understand.

Step 1: Ingestion and chunking

The corpus is the Next.js documentation, which I scraped and cleaned. Docs are a good test corpus because they mix prose, code blocks, and tables, and each of those breaks naive chunking in a different way.

RagXray lets you toggle between three chunking strategies so you can see the difference for yourself:

Naive chunking. Cut the text every 500 characters. Simple, fast, and it regularly splits a code block in half, which means retrieval later returns a chunk that starts mid-function. This is the strategy most tutorials quietly use, and it is the source of a lot of "my RAG gives weird answers" complaints.

Markdown chunking. Split on headers so each section stays together. Much better for docs, since headers are the author telling you where the topic boundaries are. The weakness is long sections that blow past your token budget.

Semantic chunking. Embed sentences with BGE-small (384 dimensions) and split where cosine similarity between neighbouring sentences drops, meaning the topic changed. This is the most expensive strategy at ingestion time, but it produces chunks that actually correspond to ideas instead of character counts.

Being able to run the same query against all three and compare the retrieved chunks side by side teaches you more about chunking in five minutes than any article, including this one.

Step 2: Embeddings, and the first roadblock

The original plan was Gemini embeddings. The free tier rate-limited my batch ingestion almost immediately, and paying per embedding for a $0 side project felt wrong.

So I skipped embedding APIs entirely and ran Hugging Face's BGE-small model natively on Cloudflare Workers AI, which executes on edge GPUs. Result: zero cost, zero rate limits, and execution fast enough that embedding happens inline with the request instead of being something you queue and wait for.

BGE-small at 384 dimensions is a deliberate trade. Bigger embedding models score better on benchmarks, but for a docs corpus the retrieval difference is small and the storage and speed difference is not. Smaller vectors mean cheaper storage in pgvector and faster similarity search.

Step 3: Retrieval, two searches in parallel

Vector search alone has a known failure mode: it is great at concepts and bad at exact tokens. Ask a vector index about getServerSideProps and it might happily return chunks about server rendering in general while missing the one chunk that contains that literal string.

Keyword search has the opposite failure mode. It nails exact terms and completely misses paraphrases.

So RagXray runs both in parallel:

Vector search goes to Neon Postgres with pgvector. Plain SQL, cosine distance, top K.

Keyword search is a custom BM25 index I wrote from scratch on Cloudflare D1. Instead of standing up a heavy search cluster like Elasticsearch for what is essentially scoring term frequencies, I store the index in SQLite tables and compute BM25 at query time. Running this on SQLite at the edge is incredibly fast, and it means the entire retrieval layer has no servers to maintain.

Writing BM25 yourself is worth doing at least once. It is about a page of code, and afterwards "keyword relevance scoring" stops being magic: it is term frequency, dampened, weighted by how rare the term is across the corpus, normalized by document length. That is the whole trick.

Step 4: Merging the two result lists with RRF

Now the awkward part. Vector search gives you cosine similarities between 0 and 1. BM25 gives you unbounded floats. You cannot just average them, and any normalization scheme you invent will be arbitrary and fragile.

The standard answer is Reciprocal Rank Fusion, and it is refreshingly dumb. RRF throws the scores away entirely and only looks at each chunk's position in each list:

score(chunk) = 1 / (60 + rank_vector) + 1 / (60 + rank_bm25)

A chunk that ranks 1st in vector search and 3rd in BM25 beats a chunk that ranks 2nd in one list and doesn't appear in the other. The constant 60 stops the top few positions from completely dominating.

That is the entire algorithm. No tuning, no normalization, no learned weights, and in practice it is very hard to beat. In RagXray you can toggle RRF off and see the merged ranking degrade in real time, which is the fastest way I know to build intuition for why hybrid retrieval works.

Step 5: Generation and self-evaluation

The top merged chunks go to Gemini 2.5 Flash to write the answer. But RagXray does one more thing that most pipelines skip: it evaluates every run, in real time, and shows you the scores.

After generating the answer, the same model scores the run on three axes:

Context relevance. Did retrieval fetch signal or noise? If this score is low, your problem is upstream of the LLM and no amount of prompt engineering will fix it.

Answer relevance. Does the answer actually address the question that was asked, or did the model answer a nearby question instead?

Faithfulness. Is every claim in the answer grounded in the retrieved context, or did the model fill gaps from its own weights? This is the hallucination check.

Splitting evaluation into these three metrics is the practical unlock. "The answer is bad" is not debuggable. "Context relevance is 0.4" tells you to go fix chunking or retrieval, not the prompt.

What building this taught me

A few things I did not fully appreciate before:

Chunking matters more than your embedding model. Most of the quality difference between runs came from how the text was split, not from anything downstream.

Hybrid retrieval is not optional for technical content. Docs are full of exact identifiers that vector search alone will fumble.

You do not need the heavy tooling. Postgres, SQLite, a small open embedding model, and one page of BM25 replace a stack that people routinely pay hundreds a month for. The managed tools earn their money at scale, but "at scale" is later than most projects think.

And evaluation should be built in from day one, not bolted on when things go wrong. If I had to keep one feature of RagXray, it would be the per-run scores.

Try it

The playground is live at ragxray-web.pages.dev. Pick a chunking strategy, ask it something about Next.js, and start toggling stages off to see what breaks.

If you are adding RAG to a real product and want a second pair of eyes on the pipeline, I do this for a living. You can reach me through salmaninayat.com.