Offline RAG Chatbot — Hackathon Platform
A fully offline, retrieval-augmented chatbot — solo-built, from vector store to generation.
- Role
- Solo-built the offline RAG chatbot end to end (ingestion, embeddings, vector search, local generation); contributed as part of the team on the shared frontend and backend.
- Timeframe
- 2026 · in progress (Hackathon 2026)
Overview
This hackathon platform is a team project built for a clinical-data-management hackathon in 2026. Within a small team, I was solely responsible for designing and building the platform’s offline chatbot: a retrieval-augmented assistant that lets users query a large corpus of clinical-data-management documents entirely offline, with no data ever leaving the local environment. I also contributed to the shared frontend and backend alongside teammates.
The problem
Clinical data management teams work with sensitive, often sponsor-owned documents that can’t be sent to a third-party AI API for search or summarization — yet teams still need fast, conversational access to what’s buried in those documents. The hackathon platform needed a chatbot that could answer questions grounded in a large, evolving document set without any external network calls, so sponsor data would never leave the environment it was ingested in.
Approach
I designed and built an offline RAG (Retrieval-Augmented Generation) pipeline end to end: documents are recursively read from a source directory, split into chunks, and embedded, with the resulting vectors stored in PostgreSQL via the pgvector extension. At query time, the user’s question is embedded with the same model, a similarity search retrieves the most relevant chunks from pgvector, and those chunks are passed as grounding context to a locally-hosted LLM for generation — no OpenAI/Anthropic/etc. API call at any point in the pipeline. That kept every sponsor document fully offline by construction, not by policy.
Architecture
- Recursive document ingestion — walks a source directory tree and reads every supported document.
- Chunking — splits ingested text into overlapping, retrieval-sized passages.
- Embedding — encodes each chunk into a vector representation.
- Vector storage & similarity search — stores embeddings in PostgreSQL via the pgvector extension; retrieves the top-matching chunks by similarity at query time.
- Local/offline generation — feeds retrieved chunks as context to a locally-hosted LLM, with no third-party API calls at any stage.
Results
Full outcomes are still being finalized as the hackathon’s judging is ongoing. The core value proposition is already proven, though: a fully offline architecture means the chatbot can run entirely inside a sponsor’s own environment with zero data egress — non-negotiable for a clinical-data-management audience.