Building a Conversational Interface for MDX Files with Bun and pgvector

Prerequisites

Setting Up PostgreSQL and pgvector

1. Install PostgreSQL: paru postgresql
2. Enable the pgvector Extension: CREATE EXTENSION vector;
3. Create the documents Table:

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    filename TEXT NOT NULL,
    chunk_index INTEGER NOT NULL,
    content TEXT NOT NULL,
    embedding VECTOR(1536) NOT NULL
);

CREATE INDEX ON documents USING HNSW (embedding vector_l2_ops);

Data Preprocessing with Bun

This stage involves reading your MDX files, segmenting their content into manageable chunks, and generating vector embeddings for each chunk.