# Les 15 — Lesopdracht ## PDF Q&A app opzetten + eerste embeddings **Vak:** AI-Assisted Development **Opleiding:** NOVI Hogeschool Utrecht **Duur:** 30 min in-class **Status:** Tijdens de les afronden — daarna huiswerk uitbreiden --- ## Doel Aan het einde van deze opdracht heb je een werkende **PDF Q&A app**: je uploadt een PDF, hij wordt gechunked + embedded + opgeslagen in Supabase pgvector, en je kunt er vragen aan stellen die accuraat beantwoord worden op basis van de PDF. --- ## Vereisten - Node 20+, pnpm, git - Supabase account - OpenAI API key --- ## Stap 1 — Nieuwe Next.js app ```bash pnpm create next-app@latest pdf-qa \ --typescript --tailwind --app --no-src-dir --import-alias "@/*" cd pdf-qa pnpm add ai @ai-sdk/openai zod @supabase/supabase-js unpdf ``` `.env.local`: ``` OPENAI_API_KEY=sk-... NEXT_PUBLIC_SUPABASE_URL=https://...supabase.co NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ... ``` --- ## Stap 2 — Supabase: pgvector + schema In Supabase SQL Editor: ```sql -- 1. Extension activeren create extension if not exists vector; -- 2. Chunks tabel create table chunks ( id bigserial primary key, source text not null, page int, content text not null, embedding vector(1536), created_at timestamp default now() ); -- 3. HNSW index voor snelle search create index on chunks using hnsw (embedding vector_cosine_ops); -- 4. RLS open voor demo alter table chunks enable row level security; create policy "demo open" on chunks for all to anon using (true) with check (true); -- 5. Match function create or replace function match_chunks( query_embedding vector(1536), match_count int default 5 ) returns table (id bigint, content text, source text, similarity float) language sql stable as $$ select chunks.id, chunks.content, chunks.source, 1 - (chunks.embedding <=> query_embedding) as similarity from chunks order by chunks.embedding <=> query_embedding limit match_count; $$; ``` --- ## Stap 3 — lib/embeddings.ts ```typescript import { embed, embedMany } from "ai"; import { openai } from "@ai-sdk/openai"; export const embedModel = openai.textEmbeddingModel("text-embedding-3-small"); export async function embedOne(text: string) { const { embedding } = await embed({ model: embedModel, value: text }); return embedding; } export async function embedBatch(texts: string[]) { const { embeddings } = await embedMany({ model: embedModel, values: texts }); return embeddings; } export function chunkText(text: string, size = 500, overlap = 50): string[] { const chunks: string[] = []; let i = 0; while (i < text.length) { chunks.push(text.slice(i, i + size).trim()); i += size - overlap; } return chunks.filter((c) => c.length > 0); } ``` --- ## Stap 4 — lib/supabase.ts ```typescript import { createClient } from "@supabase/supabase-js"; export const supabase = createClient( process.env.NEXT_PUBLIC_SUPABASE_URL!, process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY! ); ``` --- ## Stap 5 — Index endpoint `app/api/index/route.ts`: ```typescript import { extractText } from "unpdf"; import { embedBatch, chunkText } from "@/lib/embeddings"; import { supabase } from "@/lib/supabase"; export async function POST(req: Request) { const formData = await req.formData(); const file = formData.get("file") as File; if (!file) return Response.json({ error: "No file" }, { status: 400 }); const buffer = new Uint8Array(await file.arrayBuffer()); const { text } = await extractText(buffer, { mergePages: true }); const chunks = chunkText(text, 500, 50); const embeddings = await embedBatch(chunks); const { error } = await supabase.from("chunks").insert( chunks.map((content, i) => ({ source: file.name, content, embedding: embeddings[i], })) ); if (error) return Response.json({ error: error.message }, { status: 500 }); return Response.json({ chunks: chunks.length, source: file.name }); } ``` --- ## Stap 6 — Ask endpoint `app/api/ask/route.ts`: ```typescript import { embedOne } from "@/lib/embeddings"; import { supabase } from "@/lib/supabase"; import { generateText } from "ai"; import { openai } from "@ai-sdk/openai"; export async function POST(req: Request) { const { question } = await req.json(); const embedding = await embedOne(question); const { data: chunks } = await supabase.rpc("match_chunks", { query_embedding: embedding, match_count: 5, }); if (!chunks?.length) { return Response.json({ answer: "Geen relevante info gevonden." }); } const context = chunks .map((c: any, i: number) => `[Source ${i + 1}: ${c.source}]\n${c.content}`) .join("\n\n---\n\n"); const { text } = await generateText({ model: openai("gpt-4o-mini"), prompt: `Beantwoord op basis van deze context. Als geen antwoord staat, zeg dat eerlijk. CONTEXT: ${context} VRAAG: ${question} ANTWOORD:`, }); return Response.json({ answer: text, sources: chunks }); } ``` --- ## Stap 7 — Test in terminal `pnpm dev` in één terminal. In een ander: ```bash # Index een PDF (Tim deelt voorbeeld-PDF in de les) curl -X POST http://localhost:3000/api/index \ -F "file=@./polderfest-lineup.pdf" # {"chunks": 30, "source": "polderfest-lineup.pdf"} # Stel een vraag curl -X POST http://localhost:3000/api/ask \ -H "Content-Type: application/json" \ -d '{"question": "Wie zijn de headliners?"}' \ | jq ``` Verwacht: JSON met `answer` (accuraat antwoord) en `sources` (welke chunks gebruikt). --- ## Eisen - [ ] pgvector extension actief in Supabase - [ ] `chunks` tabel + HNSW index + match_chunks function - [ ] 1 PDF succesvol geïndexeerd - [ ] 3 vragen werken met correcte antwoorden - [ ] In Supabase Table Editor: chunks zichtbaar met embedding kolom --- ## Tijdsindeling (30 min) | Stap | Tijd | |------|------| | 1 — Setup | 3 min | | 2 — Supabase schema | 5 min | | 3-4 — lib files | 5 min | | 5 — Index endpoint | 7 min | | 6 — Ask endpoint | 5 min | | 7 — Test | 5 min | --- ## Veelvoorkomende problemen | Symptoom | Oplossing | |----------|-----------| | `extension "vector" does not exist` | Supabase: Database → Extensions → enable vector | | `dimension mismatch` | Embed model en `vector(N)` moeten matchen — gebruik altijd `text-embedding-3-small` (1536) | | `match_chunks does not exist` | SQL function niet aangemaakt — stap 2 stuk 5 | | `embedMany` slow | Normaal voor grote PDFs — geduld of split batches | | RLS error op insert | `create policy` niet uitgevoerd — stap 2 stuk 4 | | PDF parsing leeg | Scan-PDF (image-based)? unpdf werkt op text-PDFs | --- ## Klaar? Verder met huiswerk Het huiswerk bouwt voort: - Eigen PDF + UI + RAG-tool in een agent - `RAG.md` met analyse + fail case - Bonus: hybrid search of streaming antwoord Zie `Les15-Huiswerk.md` of `Les15-Huiswerk.pdf`.