ArabinCharts

Overview

A production-deployed application that enables natural language Q&A over PDF documents. Users upload PDFs which are chunked, embedded, and stored in pgvector. Questions are answered using retrieval-augmented generation with source attribution.

Architecture

• Document processing: PDFs are parsed, chunked with overlap, and embedded using OpenAI’s embedding model
• Vector storage: pgvector extension for PostgreSQL provides efficient similarity search
• RAG pipeline: Query embedding → vector similarity search → context assembly → LLM generation with citations
• Streaming: Server-sent events for real-time response streaming to the frontend

Key Technical Decisions

• pgvector over dedicated vector databases (Pinecone, Weaviate) — keeps the stack simple with PostgreSQL as the single data store
• Chunk overlap ensures context isn’t lost at boundary points
• Source attribution traces every answer back to specific document sections