LangGraphRAGPubMedBERTSupabasePython

MedRAG

Self-correcting RAG over biomedical papers

Upload a stack of biomedical papers and ask questions about them. MedRAG judges its own retrieval before it answers — if the passages are too thin it rewrites the query and searches again — and every claim comes back cited to a page you can open.

5-node LangGraph loop

768-d PubMedBERT

page-level citations

Stack

LangGraph · Python · FastAPI · PubMedBERT · Supabase pgvector · Pydantic · Mermaid.js

MedRAG

the problem

Ask a general-purpose RAG tool about a clinical paper and it runs one semantic search, then answers from whatever came back — even when what came back was thin. You get a confident paragraph and no page to check it against.

how it works

MedRAG runs as a LangGraph pipeline — five nodes over one shared state object, not a single call. query_analysis rewrites what you asked into something a vector index can match, retrieve embeds that with PubMedBERT and pulls the closest passages out of Supabase, and relevance_check reads them back to decide whether they're enough. Only then does synthesize write an answer. The node I care about most is relevance_check. It asks an LLM judge whether the passages actually cover the question, and if they don't it doesn't answer anyway — it routes back to query_analysis with a note about what was missing, so the next search is a different search and not the same one twice. That loop is bounded: max_query_retries caps it, and when the budget runs out MedRAG says the papers don't cover it rather than filling the gap itself.

The five nodes and the edge that matters — relevance_check can send the whole thing back to the top.
The five nodes and the edge that matters — relevance_check can send the whole thing back to the top.

citations you can check

A citation like “(Title, p. 4, Efficacy Results)” is only worth anything if page 4 really says that. So the citation pass runs after synthesis: it walks every inline citation in the response, matches it against the metadata of the chunks that were actually retrieved, and rewrites the matches into badges that link to the exact passage. Citations that don't match a retrieved chunk don't become links — if the model invented one, it stays plain text instead of getting a badge that lends it authority. Self-correcting retrieval is easy to claim, so there's a benchmark harness behind all of it, scoring Hit@K and Precision@K on retrieval, citation accuracy, and faithfulness — does the answer stay inside the excerpts it was given. Faithfulness is the one that kept me honest: it's the metric that catches the model quietly going beyond its sources.

Every badge in the answer resolves to the retrieved passage behind it.
Every badge in the answer resolves to the retrieved passage behind it.

diagram mode

Ask for a diagram instead of a paragraph and synthesize emits a Mermaid spec rather than prose. The catch is that a model writing Mermaid produces syntax that almost parses — a stray code fence, an unquoted label with a parenthesis in it — and one bad character renders nothing at all. So render_diagram cleans it before it ever reaches a renderer: strip the fences and preamble, quote every node label across all the shapes, then validate. The browser does the actual drawing, which keeps a headless Chrome off the server entirely.

Diagram mode — a sanitized Mermaid spec rendered client-side.
Diagram mode — a sanitized Mermaid spec rendered client-side.

key decisions

Judge the retrieval, not just the answer

Most guardrails check the output after the fact. Checking the passages first means a bad search gets fixed by searching again, instead of being papered over by a well-written paragraph.

PubMedBERT over a general embedding API

Drug names, trial acronyms, and clinical jargon are exactly where general-purpose embeddings blur together. A domain model runs locally, costs nothing per token, and separates the terms that matter here.

Sanitize on the server, render in the browser

The backend owns correctness — cleaning and validating the Mermaid spec. The browser owns pixels. That split keeps Puppeteer and a headless Chrome out of the deployment entirely.

Stateless server, state in the browser

Threads, uploaded PDFs, and history live in localStorage; each request carries the document ids it should search. The server stays trivial to scale and retrieval stays scoped to the papers you're actually reading.

outcomes

  • Bounded self-correction — thin retrievals trigger a rewritten query instead of a confident guess
  • 768-d PubMedBERT embeddings over Supabase pgvector, scoped per conversation
  • Citations resolve to a page, and unmatched ones never render as links
  • Mermaid specs validated server-side before they reach the renderer
  • Eval harness scoring Hit@K, Precision@K, faithfulness, and citation accuracy
  • Open-sourced on GitHub

up next

NeuroBridge

Clinician-gated AI exercise agent for upper-limb rehab