fix: les
This commit is contained in:
1
Les16-RAG-Embeddings/.~lock.Les14-Slides.pdf#
Normal file
1
Les16-RAG-Embeddings/.~lock.Les14-Slides.pdf#
Normal file
@@ -0,0 +1 @@
|
||||
,sharp-zealous-planck,claude,21.05.2026 10:59,file:///sessions/sharp-zealous-planck/.config/libreoffice/4;
|
||||
575
Les16-RAG-Embeddings/Les16-Docenttekst.md
Normal file
575
Les16-RAG-Embeddings/Les16-Docenttekst.md
Normal file
@@ -0,0 +1,575 @@
|
||||
# Les 16 — RAG + Embeddings
|
||||
## Docenttekst (Klas A — 3 uur, fysiek, demo-driven)
|
||||
|
||||
**Les:** 15 van 18
|
||||
**Onderwerp:** RAG (Retrieval-Augmented Generation) — AI laten antwoorden op eigen documenten
|
||||
**Duur:** 180 minuten
|
||||
**Format:** Tim demonstreert klassikaal. Studenten kijken mee. Zelf bouwen = huiswerk.
|
||||
**Demo-app:** PDF Q&A from scratch — nieuwe kleine app
|
||||
|
||||
---
|
||||
|
||||
## Hoe deze tekst werkt
|
||||
|
||||
- `[SLIDE X]` — klik naar slide X
|
||||
- `[SCHERM: slides | terminal | editor | browser | supabase]`
|
||||
- **Vertel:** "..." — wat je zegt
|
||||
- `*[stage direction]*` — instructie voor jezelf
|
||||
- 💬 = verwachte studentenvraag
|
||||
|
||||
---
|
||||
|
||||
## VÓÓR DE LES — Setup (45 min)
|
||||
|
||||
### 1. Demo-folder + Supabase
|
||||
|
||||
- Maak vooraf werkende `pdf-qa` lokaal (backup)
|
||||
- Supabase project: pgvector extension AL geactiveerd
|
||||
- Schema (chunks tabel + match_chunks function) al gedraaid
|
||||
- 1 PDF al klaar om te indexeren: `polderfest-lineup-2027.pdf` (genereer met AI of pak echte)
|
||||
|
||||
### 2. Voorbeeld-PDF klaar
|
||||
|
||||
- Print of genereer een PDF over Polderfest 2027 — ~10-20 pagina's
|
||||
- Liefst: line-up, dagschema's, locatie-info
|
||||
- Studenten kunnen deze ook downloaden
|
||||
- Backup-PDF: Wikipedia-artikel over willekeurig onderwerp als geprinte PDF
|
||||
|
||||
### 3. Browser-tabs
|
||||
|
||||
- `localhost:3000` (dev server)
|
||||
- Supabase dashboard (Table Editor + SQL Editor)
|
||||
- https://ai-sdk.dev/docs/ai-sdk-core/embeddings (referentie)
|
||||
- https://platform.openai.com/usage (cost-monitoring)
|
||||
|
||||
### 4. Backup als demo crasht
|
||||
|
||||
- Werkende eindstaat (alle code) op USB
|
||||
- Console-log met "voorbeeld-embedding" als slide-content
|
||||
- Backup-vraag-en-antwoord screenshots
|
||||
|
||||
### 5. Terminals
|
||||
|
||||
- 3 tabs: dev, watch (`watch curl ...`), supabase SQL via CLI als nodig
|
||||
|
||||
---
|
||||
|
||||
# HET SCRIPT — Lees mee tijdens de les
|
||||
|
||||
## BLOK 1 — Welkom + Terugblik (10 min)
|
||||
|
||||
`[SLIDE 1 — Title]` `[SCHERM: slides]`
|
||||
|
||||
**Vertel:** "Welkom bij les 15. Vandaag: RAG. Retrieval-Augmented Generation. AI laten antwoorden op basis van jouw eigen documenten."
|
||||
|
||||
`[SLIDE 2 — Terugblik]`
|
||||
|
||||
**Vertel:** "Lessen 11-14: AI SDK basics, tool calling, agents, externe APIs en deploy. Nu hebben we een complete stack.
|
||||
|
||||
Wat we nog niet hadden: AI laten werken met **jouw documenten**. Stel je hebt 200 pagina's productdocumentatie of een dik handboek of een lange whitepaper. Hoe stel je daar vragen aan?
|
||||
|
||||
Twee naïeve oplossingen. Eén: alles meesturen in context. Werkt voor 5 pagina's, niet voor 500. Te duur. Twee: AI zelf laten zoeken. Werkt voor publieke info, niet voor jouw private data.
|
||||
|
||||
RAG is de derde oplossing. In één zin: haal de relevante stukjes uit je documenten op, geef die aan AI, AI antwoordt."
|
||||
|
||||
`[SLIDE 3 — Planning]`
|
||||
|
||||
**Vertel:** "Drie uur. Eerst vijftig minuten theorie — embeddings, similarity, pipeline, pgvector, chunking. Veel om te begrijpen, maar het loont. Daarna vier demos. Pauze rond minuut 100."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 2 — Theorie embeddings (20 min)
|
||||
|
||||
`[SLIDE 4 — Wat is een embedding]`
|
||||
|
||||
**Vertel:** "Een embedding is een vector. Letterlijk een array van getallen. Voor OpenAI's small model: 1536 getallen tussen min-één en plus-één.
|
||||
|
||||
Je stopt tekst erin, krijgt vector terug. Maar het magische zit hier — vergelijkbare betekenissen produceren vergelijkbare vectors.
|
||||
|
||||
`*[Wijs naar slide voorbeeld]*`
|
||||
|
||||
'De kat zit op de mat' en 'Een poes ligt op het tapijt' hebben bijna geen woorden gemeen. Maar betekenis is hetzelfde. Hun embeddings staan dichtbij elkaar in de 1536-dimensionale ruimte. 'Voetbal in Nederland' — totaal andere kant op.
|
||||
|
||||
Dit is fundamenteel anders dan keyword search. Keyword zou de eerste twee zinnen totaal niet matchen — geen woord overlap. Semantic search wel.
|
||||
|
||||
Welke modellen. OpenAI 3-small en 3-large — de standaarden. Open-source alternatieven zoals nomic-embed-text als je lokaal wilt embedden. Voor vandaag: 3-small. Goedkoop, snel, prima kwaliteit voor de meeste apps."
|
||||
|
||||
💬 *Vraag: 'Wat doet zo'n embedding-model eigenlijk?'*
|
||||
|
||||
**Antwoord:** "Het is een neural network, getraind om de betekenis van tekst te 'condenseren' naar een dichte vector. Vergelijkbaar met de eerste lagen van een GPT-model, maar zonder de generative head. Heel veel context distilleren tot 1536 nummers. Wiskundig: een functie die tekst projecteert in een 'semantic space'."
|
||||
|
||||
`[SLIDE 5 — Vector similarity]`
|
||||
|
||||
**Vertel:** "Hoe meet je 'dichtbij'? Drie metrics, één winnaar voor text.
|
||||
|
||||
Cosine similarity — hoek tussen twee vectors. Van min-één tot plus-één. Plus-één betekent identiek georiënteerd, nul ongerelateerd, min-één tegenovergesteld. Voor genormaliseerde vectors (OpenAI's zijn dat) is cosine effectief dot product.
|
||||
|
||||
Voor RAG: cosine. Standaard, robuust, simpel.
|
||||
|
||||
In pgvector schrijf je `<=>` voor cosine distance. Een kleinere afstand = meer similar. Order by `<=>` ascending = top resultaten eerst. Onthoud die operator."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 3 — Theorie RAG pipeline + pgvector (30 min)
|
||||
|
||||
`[SLIDE 6 — RAG pipeline]`
|
||||
|
||||
**Vertel:** "Twee pipelines. Index time en query time.
|
||||
|
||||
Index time gebeurt eenmalig — wanneer je een document uploadt. PDF binnen, parsen, opknippen in chunks van ongeveer vijfhonderd tokens, elke chunk embedden, alles in Postgres.
|
||||
|
||||
Query time gebeurt elke vraag. Vraag binnen, embedden — zelfde model als index, dat is belangrijk — similarity search in Postgres, top vijf chunks terug. Die geef je als context aan LLM, LLM antwoordt.
|
||||
|
||||
Belangrijkste inzicht: LLM ziet nooit de hele DB. Alleen de vijf meest relevante chunks. Schaalbaar tot miljoenen documenten."
|
||||
|
||||
💬 *Vraag: 'Waarom moet het embedding-model hetzelfde zijn?'*
|
||||
|
||||
**Antwoord:** "Embeddings zijn alleen vergelijkbaar binnen hetzelfde model. Model A's vector voor 'kat' staat op een totaal andere plek dan Model B's vector voor 'kat'. Hun coördinatensystemen zijn niet uitwisselbaar. Dus: zelfde model voor index én query — always."
|
||||
|
||||
`[SLIDE 7 — pgvector]`
|
||||
|
||||
**Vertel:** "Hoe sla je vectors op? Postgres heeft een extensie: pgvector. Voegt `vector` datatype toe, plus operators voor cosine, euclidean, dot product. In Supabase: één klik in dashboard om te activeren.
|
||||
|
||||
`*[Toon SQL op slide]*`
|
||||
|
||||
Schema: chunks tabel met content kolom en embedding kolom. `vector(1536)` — dimensie matcht model. Voor performance: een HNSW index. Hierarchical Navigable Small Worlds. Wiskundige magie waardoor je in miljoenen vectors in milliseconden zoekt. Bij minder dan tienduizend rows kun je het skippen — exacte search is dan al snel."
|
||||
|
||||
`[SLIDE 8 — Chunking]`
|
||||
|
||||
**Vertel:** "Hoe knip je een document op. Klinkt simpel — is het niet.
|
||||
|
||||
Drie strategieën. Fixed size — vijfhonderd tokens per chunk, vijftig tokens overlap. Simpel, default. Recursive — knip eerst op paragrafen, dan zinnen, dan woorden. Behoudt structuur beter. Semantic — embed zinnen, group similar. Beste kwaliteit, duurder.
|
||||
|
||||
Sweet spot voor chunks: tweehonderd tot vijfhonderd tokens. Te klein, je verliest context. Te groot, irrelevant info verdunt het signaal. Overlap van tien tot vijftien procent zodat info aan chunk-grenzen niet verdwijnt.
|
||||
|
||||
Vandaag: fixed size, vijfhonderd char chunks, vijftig overlap. Werkt prima voor de meeste docs."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 4 — Demo 1: pgvector setup (25 min)
|
||||
|
||||
`[SLIDE 9 — Wat we bouwen]`
|
||||
|
||||
**Vertel:** "Vandaag bouwen we een PDF Q&A app from scratch. Upload PDF, hij wordt geïndexeerd, je stelt vragen. Klassieke RAG-use-case."
|
||||
|
||||
`[SLIDE 10 — DEMO 1]` `[SCHERM: terminal + editor]`
|
||||
|
||||
**Vertel:** "Klas, kijk mee."
|
||||
|
||||
```bash
|
||||
cd ~/novi/novi-lessons/Les16-RAG-Embeddings
|
||||
pnpm create next-app@latest pdf-qa \
|
||||
--typescript --tailwind --app --no-src-dir --import-alias "@/*"
|
||||
cd pdf-qa
|
||||
pnpm add ai @ai-sdk/openai zod @supabase/supabase-js unpdf
|
||||
cursor .
|
||||
```
|
||||
|
||||
`*[Supabase dashboard]*` `[SCHERM: supabase]`
|
||||
|
||||
**Vertel:** "Eerst pgvector activeren. Database → Extensions → zoek vector → enable. Eén click.
|
||||
|
||||
Of via SQL:"
|
||||
|
||||
```sql
|
||||
create extension if not exists vector;
|
||||
```
|
||||
|
||||
`*[SQL Editor — voer uit:]*`
|
||||
|
||||
```sql
|
||||
create table chunks (
|
||||
id bigserial primary key,
|
||||
source text not null,
|
||||
page int,
|
||||
content text not null,
|
||||
embedding vector(1536),
|
||||
created_at timestamp default now()
|
||||
);
|
||||
|
||||
create index on chunks
|
||||
using hnsw (embedding vector_cosine_ops);
|
||||
|
||||
alter table chunks enable row level security;
|
||||
create policy "demo open" on chunks
|
||||
for all to anon using (true) with check (true);
|
||||
```
|
||||
|
||||
**Vertel:** "Tabel, HNSW index, RLS, open policy. Dezelfde regels als alle eerdere demos."
|
||||
|
||||
`*[Editor → lib/embeddings.ts]*`
|
||||
|
||||
```typescript
|
||||
import { embed, embedMany } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
export const embedModel = openai.textEmbeddingModel("text-embedding-3-small");
|
||||
|
||||
export async function embedOne(text: string) {
|
||||
const { embedding } = await embed({ model: embedModel, value: text });
|
||||
return embedding;
|
||||
}
|
||||
|
||||
export async function embedBatch(texts: string[]) {
|
||||
const { embeddings } = await embedMany({ model: embedModel, values: texts });
|
||||
return embeddings;
|
||||
}
|
||||
```
|
||||
|
||||
**Vertel:** "Twee functies. embedOne voor één string, embedBatch voor veel tegelijk. Batch is veel sneller — één API-call in plaats van een loop."
|
||||
|
||||
`*[Voeg chunkText functie toe]*`
|
||||
|
||||
```typescript
|
||||
export function chunkText(text: string, size = 500, overlap = 50): string[] {
|
||||
const chunks: string[] = [];
|
||||
let i = 0;
|
||||
while (i < text.length) {
|
||||
chunks.push(text.slice(i, i + size).trim());
|
||||
i += size - overlap;
|
||||
}
|
||||
return chunks.filter((c) => c.length > 0);
|
||||
}
|
||||
```
|
||||
|
||||
**Vertel:** "Simpelste chunker mogelijk. Slice op chars. Voor productie: gebruik LangChain's RecursiveCharacterTextSplitter — handelt edge cases beter. Voor demo: dit volstaat."
|
||||
|
||||
`*[Test embed]*`
|
||||
|
||||
```typescript
|
||||
// Eenmalig in een test-script of API route
|
||||
const e = await embedOne("De kat zit op de mat");
|
||||
console.log(e.length, e.slice(0, 5));
|
||||
// 1536 [0.012, -0.0345, 0.0123, ...]
|
||||
```
|
||||
|
||||
**Vertel:** "Vector van 1536 nummers. Eerste vijf: kleine getallen tussen min-één en plus-één. Dat is alles."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 5 — Demo 2: Index pipeline (20 min)
|
||||
|
||||
`[SLIDE 11 — DEMO 2]` `[SCHERM: editor + terminal]`
|
||||
|
||||
**Vertel:** "Index endpoint. POST een PDF erin, krijg chunk-count terug."
|
||||
|
||||
`*[app/api/index/route.ts]*`
|
||||
|
||||
```typescript
|
||||
import { extractText } from "unpdf";
|
||||
import { embedBatch, chunkText } from "@/lib/embeddings";
|
||||
import { supabase } from "@/lib/supabase";
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const formData = await req.formData();
|
||||
const file = formData.get("file") as File;
|
||||
if (!file) return Response.json({ error: "No file" }, { status: 400 });
|
||||
|
||||
const buffer = new Uint8Array(await file.arrayBuffer());
|
||||
const { text } = await extractText(buffer, { mergePages: true });
|
||||
|
||||
const chunks = chunkText(text, 500, 50);
|
||||
const embeddings = await embedBatch(chunks);
|
||||
|
||||
const { error } = await supabase.from("chunks").insert(
|
||||
chunks.map((content, i) => ({
|
||||
source: file.name,
|
||||
content,
|
||||
embedding: embeddings[i],
|
||||
}))
|
||||
);
|
||||
|
||||
if (error) return Response.json({ error: error.message }, { status: 500 });
|
||||
return Response.json({ chunks: chunks.length, source: file.name });
|
||||
}
|
||||
```
|
||||
|
||||
**Vertel:** "Vier stappen. Eén: ontvang file via formData. Twee: parse met unpdf. Drie: chunk + embed. Vier: insert alles in Supabase."
|
||||
|
||||
`*[Terminal:]*`
|
||||
|
||||
```bash
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
`*[Tweede terminal:]*`
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/index \
|
||||
-F "file=@/path/to/polderfest-lineup.pdf"
|
||||
```
|
||||
|
||||
`*[Wacht 5-10s]*`
|
||||
|
||||
**Vertel:** "Response: chunks: 30. Dertig chunks van onze PDF in de DB."
|
||||
|
||||
`*[Supabase Table Editor]*` `[SCHERM: supabase]`
|
||||
|
||||
**Vertel:** "Daar staan ze. Content kolom met tekst. Embedding kolom — clickbaar voor de vector. Source filename. Klaar voor querying."
|
||||
|
||||
💬 *Vraag: 'Hoe lang duurt embedding van 1000 chunks?'*
|
||||
|
||||
**Antwoord:** "Met embedMany batch: enkele seconden voor duizend chunks. OpenAI's API is snel — duizenden tokens per seconde. Voor één PDF van 200 pagina's: misschien 5-10 seconden totaal. Indexing is goedkoop én snel."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 6 — Pauze (15 min)
|
||||
|
||||
`[SLIDE 12 — Pauze]`
|
||||
|
||||
---
|
||||
|
||||
## BLOK 7 — Demo 3: Query pipeline (20 min)
|
||||
|
||||
`[SLIDE 13 — DEMO 3]` `[SCHERM: editor + terminal + supabase]`
|
||||
|
||||
**Vertel:** "Nu querying. We schrijven een SQL function voor de similarity search."
|
||||
|
||||
`*[Supabase SQL Editor:]*`
|
||||
|
||||
```sql
|
||||
create or replace function match_chunks(
|
||||
query_embedding vector(1536),
|
||||
match_count int default 5
|
||||
)
|
||||
returns table (id bigint, content text, source text, similarity float)
|
||||
language sql stable as $$
|
||||
select chunks.id, chunks.content, chunks.source,
|
||||
1 - (chunks.embedding <=> query_embedding) as similarity
|
||||
from chunks
|
||||
order by chunks.embedding <=> query_embedding
|
||||
limit match_count;
|
||||
$$;
|
||||
```
|
||||
|
||||
**Vertel:** "Postgres function neemt vector + count, returnt top N met similarity score. `1 - (a <=> b)` converteert distance terug naar similarity score."
|
||||
|
||||
`*[app/api/ask/route.ts]*`
|
||||
|
||||
```typescript
|
||||
import { embedOne } from "@/lib/embeddings";
|
||||
import { supabase } from "@/lib/supabase";
|
||||
import { generateText } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const { question } = await req.json();
|
||||
const embedding = await embedOne(question);
|
||||
|
||||
const { data: chunks } = await supabase.rpc("match_chunks", {
|
||||
query_embedding: embedding,
|
||||
match_count: 5,
|
||||
});
|
||||
|
||||
if (!chunks?.length) {
|
||||
return Response.json({ answer: "Geen relevante info gevonden." });
|
||||
}
|
||||
|
||||
const context = chunks
|
||||
.map((c: any, i: number) => `[Source ${i + 1}: ${c.source}]\n${c.content}`)
|
||||
.join("\n\n---\n\n");
|
||||
|
||||
const { text } = await generateText({
|
||||
model: openai("gpt-4o-mini"),
|
||||
prompt: `Beantwoord de vraag op basis van deze context. Als geen antwoord in de context staat, zeg dat eerlijk.
|
||||
|
||||
CONTEXT:
|
||||
${context}
|
||||
|
||||
VRAAG: ${question}
|
||||
|
||||
ANTWOORD:`,
|
||||
});
|
||||
|
||||
return Response.json({ answer: text, sources: chunks });
|
||||
}
|
||||
```
|
||||
|
||||
**Vertel:** "Vier stappen. Eén: embed de vraag. Twee: RPC call naar match_chunks — top 5 terug. Drie: bouw context-string met source-labels. Vier: generateText met prompt-template."
|
||||
|
||||
`*[Terminal:]*`
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/ask \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"question": "Wie zijn de headliners op Polderfest 2027?"}' \
|
||||
| jq
|
||||
```
|
||||
|
||||
`*[Wacht 2-3s]*`
|
||||
|
||||
**Vertel:** "Antwoord: 'De headliners op Polderfest 2027 zijn... blah blah, gebaseerd op de line-up uit het document.'
|
||||
|
||||
Plus: sources. Welke chunks waren de top vijf. Similarity scores. Volledig traceerbaar."
|
||||
|
||||
`*[Tweede vraag voor demo:]*`
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/ask \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"question": "Op welk podium spelen de jazz acts?"}' \
|
||||
| jq
|
||||
```
|
||||
|
||||
**Vertel:** "Andere vraag, andere chunks opgehaald, ander antwoord. Werkt."
|
||||
|
||||
💬 *Vraag: 'Wat als vraag niet in document staat?'*
|
||||
|
||||
**Antwoord:** "Twee dingen. Eén: similarity scores zijn laag — top vijf is alsnog niet relevant. Twee: LLM merkt dat context het niet bevat, en zegt 'staat niet in document'. Dat tweede dwingen we af met de prompt: 'Als geen antwoord in context staat, zeg dat eerlijk.' Anders zou-ie kunnen hallucineren."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 8 — Demo 4: RAG-tool in agent (15 min)
|
||||
|
||||
`[SLIDE 14 — DEMO 4]` `[SCHERM: editor]`
|
||||
|
||||
**Vertel:** "Laatste demo. Combineer Les 13 met Les 16. Een RAG-tool in een ToolLoopAgent."
|
||||
|
||||
`*[lib/agent.ts]*`
|
||||
|
||||
```typescript
|
||||
import { ToolLoopAgent, tool, stepCountIs } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
import { embedOne } from "./embeddings";
|
||||
import { supabase } from "./supabase";
|
||||
import { z } from "zod";
|
||||
|
||||
const ragSearch = tool({
|
||||
description:
|
||||
"Zoek in geüploade documenten op basis van semantic similarity. " +
|
||||
"Gebruik voor inhoudelijke vragen over de documenten.",
|
||||
inputSchema: z.object({
|
||||
query: z.string().describe("Wat je wilt vinden"),
|
||||
}),
|
||||
execute: async ({ query }) => {
|
||||
const embedding = await embedOne(query);
|
||||
const { data } = await supabase.rpc("match_chunks", {
|
||||
query_embedding: embedding,
|
||||
match_count: 5,
|
||||
});
|
||||
return data;
|
||||
},
|
||||
});
|
||||
|
||||
export const docAgent = new ToolLoopAgent({
|
||||
model: openai("gpt-4o-mini"),
|
||||
system: `Je beantwoordt vragen over geüploade documenten. Werkwijze:
|
||||
1. Zoek met ragSearch voor relevante info
|
||||
2. Lees de chunks
|
||||
3. Eventueel: tweede ragSearch met andere query voor meer context
|
||||
4. Antwoord met bronvermeldingen.
|
||||
|
||||
Verzin niets - alleen wat in de chunks staat.`,
|
||||
tools: { ragSearch },
|
||||
stopWhen: stepCountIs(10),
|
||||
});
|
||||
```
|
||||
|
||||
**Vertel:** "Agent met één tool: ragSearch. System prompt vertelt hem multi-step te werken — zoek, lees, evalueer of nog een query nodig is.
|
||||
|
||||
Het voordeel boven simple RAG: voor complexe vragen kan agent meerdere searches doen. 'Vergelijk de jazz- en rock-headliners' — agent doet twee searches, één voor jazz, één voor rock, dan vergelijkt."
|
||||
|
||||
`*[Quick test in script of route:]*`
|
||||
|
||||
```typescript
|
||||
const result = await docAgent.generate({
|
||||
prompt: "Vergelijk de jazz- en rock-headliners op Polderfest 2027.",
|
||||
});
|
||||
console.log(result.text);
|
||||
console.log("Steps:", result.steps.length);
|
||||
```
|
||||
|
||||
`*[Run]*`
|
||||
|
||||
**Vertel:** "Antwoord: vergelijking van beide. Steps: drie of vier — twee searches, dan reasoning, dan finale text. Agent doet zijn werk."
|
||||
|
||||
💬 *Vraag: 'Wanneer agent vs simple RAG?'*
|
||||
|
||||
**Antwoord:** "Simple RAG voor single-shot vragen. 'Wie is X?' → één search → antwoord. Agent voor vragen die multiple lookups vereisen. Vergelijkingen, multi-hop, of als gebruiker breed vraagt en je wilt dat AI verfijnt. Trade-off: agent kost 3-5x meer en duurt langer. Begin met simple, upgrade als nodig."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 9 — Wanneer RAG (10 min)
|
||||
|
||||
`[SLIDE 15 — Wanneer wel/niet]`
|
||||
|
||||
**Vertel:** "Niet alles is RAG. Wanneer wel: veel documenten, ze veranderen, semantic search nodig, privacy belangrijk, bronvermelding gewenst.
|
||||
|
||||
Wanneer niet: klein document — stop in prompt. Exacte data — gebruik tool-calls of SQL. Structured data — SQL is beter dan vector search. Code-base — gebruik grep en tree-sitter.
|
||||
|
||||
In productie zie je vaak hybrid. Semantic search voor concept-vragen, keyword search voor exacte termen, metadata filter voor structured criteria. Drie technieken in één query.
|
||||
|
||||
En: re-ranking. Top-twintig ophalen via vector, dan re-rank met Cohere of LLM-as-judge naar top-vijf. Twintig procent betere kwaliteit. Voor productie: doen. Voor demo: skip."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 10 — Lesopdracht + Huiswerk (10 min)
|
||||
|
||||
`[SLIDE 16 — Lesopdracht + Huiswerk]`
|
||||
|
||||
**Vertel:** "Lesopdracht: zelfde app als ik liet zien. Half uur. PDF Q&A, pgvector, index, ask. Voorbeeld-PDF deel ik in de groep.
|
||||
|
||||
Huiswerk: drie dingen.
|
||||
|
||||
Eén: echte PDF van minimaal twintig pagina's, eigen interesse. UI met file upload + chat-interface.
|
||||
|
||||
Twee: RAG-tool in een ToolLoopAgent. UI laat tool-invocations zien zoals Les 12.
|
||||
|
||||
Drie: RAG.md schrijven. Document-info, vijf werkende vragen met sources, één fail-case waar RAG het verkeerd had, één chunking-experiment, één observatie.
|
||||
|
||||
Bonus: hybrid search, streaming met citations, re-ranking, multi-document.
|
||||
|
||||
Tien punten, voldoende zes."
|
||||
|
||||
---
|
||||
|
||||
## BLOK 11 — Afsluiting (5 min)
|
||||
|
||||
`[SLIDE 17 — Afsluiting]`
|
||||
|
||||
**Vertel:** "Wat hebben we vandaag gedaan. Embeddings als vectors. Cosine similarity. RAG pipeline — index + query. pgvector. Chunking. RAG-tool in een agent. Wanneer wel/niet RAG.
|
||||
|
||||
Volgende les: Multimodal. Voice, vision, image generation. Whisper voor transcriptie. GPT-4o voor foto-analyse. Flux of DALL-E voor afbeeldingen genereren in je app. Heel visueel.
|
||||
|
||||
Daarna: les 17 over MCP — Model Context Protocol — eigen MCP server bouwen. Les 18: browser automation en Computer Use. Spannende lessen om mee af te sluiten.
|
||||
|
||||
Vragen?"
|
||||
|
||||
`*[Vragenronde — minstens 5 min over laten]*`
|
||||
|
||||
---
|
||||
|
||||
## NA DE LES — Wrap-up
|
||||
|
||||
- Push working `pdf-qa` repo naar GitHub als referentie
|
||||
- Deel voorbeeld-PDF voor lesopdracht in Brightspace
|
||||
- Brightspace: huiswerk-instructies + repo URL
|
||||
- Voor Klas B: check of zelfde stack werkt (Supabase free tier limiet pgvector?)
|
||||
|
||||
---
|
||||
|
||||
## Veelvoorkomende fouten tijdens live coding
|
||||
|
||||
| Fout | Oplossing |
|
||||
|------|-----------|
|
||||
| `extension "vector" does not exist` | Supabase: enable in dashboard |
|
||||
| Embedding dimension mismatch | Check `vector(1536)` matcht `text-embedding-3-small` |
|
||||
| `match_chunks function not found` | SQL function niet aangemaakt — kopieer uit slide |
|
||||
| RLS error op insert | Open policy nog niet uitgevoerd |
|
||||
| `unpdf` geen text uit PDF | Image-based PDF (scan) — gebruik OCR of andere PDF |
|
||||
| HNSW index trager dan exact | Voor <10k rows: skip index of `set hnsw.ef_search = 100` |
|
||||
| Antwoord is generic, mist details | Top-k te laag of chunks te klein — experimenteer |
|
||||
| Hallucination | Strengere prompt: "Als info ontbreekt, zeg dat" |
|
||||
|
||||
---
|
||||
|
||||
## Mentale model voor de klas
|
||||
|
||||
Als studenten verward zijn over wat embeddings doen:
|
||||
|
||||
> Een embedding is een **adres** in de betekenis-ruimte. Twee adressen dichtbij elkaar = twee zinnen met vergelijkbare betekenis. RAG werkt door: vertaal vraag naar adres, vind alle adressen in de buurt, lees wat daar woont, geef dat aan AI.
|
||||
|
||||
En RAG vs Agent:
|
||||
|
||||
> RAG = bibliotheek met index. Eén vraag, één bezoek, top-vijf boeken, antwoord.
|
||||
> RAG-in-Agent = bibliothecaris die voor jou zoekt. Stelt eerst sub-vragen, doet meerdere zoekrondes, vergelijkt, antwoordt.
|
||||
257
Les16-RAG-Embeddings/Les16-Huiswerk.md
Normal file
257
Les16-RAG-Embeddings/Les16-Huiswerk.md
Normal file
@@ -0,0 +1,257 @@
|
||||
# Les 15 — Huiswerk
|
||||
## Eigen PDF + simpele chat-UI + RAG.md
|
||||
|
||||
**Vak:** AI-Assisted Development
|
||||
**Opleiding:** NOVI Hogeschool Utrecht
|
||||
**Deadline:** Voor Les 16 — MCP servers
|
||||
**Inleveren:** GitHub repo + `RAG.md` in root
|
||||
|
||||
---
|
||||
|
||||
## Doel
|
||||
|
||||
Bouwt voort op de **lesopdracht** (werkende PDF Q&A app met 1 PDF). In dit huiswerk:
|
||||
|
||||
- Index een echte PDF van minimaal 20 pagina's
|
||||
- Maak een simpele upload + chat-UI
|
||||
- Documenteer 5 vragen + 1 fail-case in `RAG.md`
|
||||
|
||||
---
|
||||
|
||||
## Onderdeel A — Echte PDF + UI (verplicht)
|
||||
|
||||
### Stap 1 — Kies een PDF
|
||||
|
||||
Kies een PDF van minimaal 20 pagina's, iets dat jou interesseert:
|
||||
- Studieboek-hoofdstuk
|
||||
- Open-source whitepaper (LLaMA, RAG-papers)
|
||||
- Bedrijfsjaarverslag
|
||||
- Wikipedia-artikel als PDF geprint
|
||||
- Game manual
|
||||
|
||||
Moet text-based zijn (geen scan).
|
||||
|
||||
### Stap 2 — Upload-UI
|
||||
|
||||
`app/page.tsx` — file upload + lijst van geïndexeerde docs:
|
||||
|
||||
```tsx
|
||||
"use client";
|
||||
import { useState } from "react";
|
||||
|
||||
export default function Page() {
|
||||
const [file, setFile] = useState<File | null>(null);
|
||||
const [indexing, setIndexing] = useState(false);
|
||||
const [result, setResult] = useState<string>("");
|
||||
|
||||
async function handleUpload() {
|
||||
if (!file) return;
|
||||
setIndexing(true);
|
||||
const fd = new FormData();
|
||||
fd.append("file", file);
|
||||
const res = await fetch("/api/index", { method: "POST", body: fd });
|
||||
const data = await res.json();
|
||||
setResult(`${data.chunks} chunks geïndexeerd`);
|
||||
setIndexing(false);
|
||||
}
|
||||
|
||||
return (
|
||||
<main className="max-w-2xl mx-auto p-8">
|
||||
<h1 className="text-3xl font-bold mb-6">PDF Q&A</h1>
|
||||
<input type="file" accept=".pdf"
|
||||
onChange={(e) => setFile(e.target.files?.[0] ?? null)} />
|
||||
<button onClick={handleUpload} disabled={indexing}
|
||||
className="ml-3 bg-blue-600 text-white px-4 py-2 rounded">
|
||||
{indexing ? "..." : "Index"}
|
||||
</button>
|
||||
{result && <p className="mt-4 text-green-600">{result}</p>}
|
||||
|
||||
<ChatBox />
|
||||
</main>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Stap 3 — Chat-interface (simpel)
|
||||
|
||||
Geen useChat / streaming nodig — gewoon een form + lijst van vragen+antwoorden:
|
||||
|
||||
```tsx
|
||||
"use client";
|
||||
import { useState } from "react";
|
||||
|
||||
export function ChatBox() {
|
||||
const [q, setQ] = useState("");
|
||||
const [history, setHistory] = useState<{q: string; a: string}[]>([]);
|
||||
const [loading, setLoading] = useState(false);
|
||||
|
||||
async function ask() {
|
||||
if (!q.trim()) return;
|
||||
setLoading(true);
|
||||
const res = await fetch("/api/ask", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ question: q }),
|
||||
});
|
||||
const data = await res.json();
|
||||
setHistory((h) => [...h, { q, a: data.answer }]);
|
||||
setQ("");
|
||||
setLoading(false);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="mt-8">
|
||||
<div className="flex gap-2 mb-4">
|
||||
<input value={q} onChange={(e) => setQ(e.target.value)}
|
||||
placeholder="Stel een vraag..."
|
||||
className="flex-1 border p-2 rounded" />
|
||||
<button onClick={ask} disabled={loading}
|
||||
className="bg-blue-600 text-white px-4 py-2 rounded">
|
||||
{loading ? "..." : "Vraag"}
|
||||
</button>
|
||||
</div>
|
||||
<ul className="space-y-3">
|
||||
{history.map((item, i) => (
|
||||
<li key={i} className="border rounded p-3">
|
||||
<p className="font-semibold">Q: {item.q}</p>
|
||||
<p className="text-gray-700 mt-1">{item.a}</p>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Eisen
|
||||
|
||||
- [ ] Upload werkt — file input + index endpoint call
|
||||
- [ ] Geïndexeerde docs zichtbaar (chunks count)
|
||||
- [ ] Chat-interface werkt (vraag/antwoord)
|
||||
- [ ] Antwoorden zijn duidelijk gebaseerd op PDF-inhoud
|
||||
|
||||
---
|
||||
|
||||
## Onderdeel B — `RAG.md` documentatie (verplicht)
|
||||
|
||||
Schrijf in repo-root.
|
||||
|
||||
### Sectie 1 — Document info
|
||||
|
||||
- Bestandsnaam + onderwerp
|
||||
- Aantal pagina's
|
||||
- Aantal chunks na indexing
|
||||
- Chunk-size + overlap die je gebruikt hebt
|
||||
|
||||
### Sectie 2 — 5 succesvolle vragen
|
||||
|
||||
Voor elk:
|
||||
|
||||
```markdown
|
||||
**Vraag 1:** "Wat zijn de drie hoofdkenmerken van X?"
|
||||
|
||||
**Antwoord (samenvatting):** ...
|
||||
|
||||
**Top chunks (top 2-3):**
|
||||
- Source page 12 — "X heeft drie kenmerken..."
|
||||
- Source page 14 — "Het derde kenmerk..."
|
||||
```
|
||||
|
||||
### Sectie 3 — 1 fail-case
|
||||
|
||||
Een vraag waar RAG het **niet goed deed**:
|
||||
|
||||
```markdown
|
||||
**Vraag:** "Wat staat er over Y?"
|
||||
|
||||
**Antwoord (incorrect):** "Y wordt niet besproken in het document."
|
||||
|
||||
**Wat ging mis:** Y wordt wel besproken op pagina 8, maar onder een ander
|
||||
woord (Z). Embedding van "Y" was te ver van embedding van paragraaf over "Z".
|
||||
|
||||
**Wat zou helpen:** Synoniem-expansion in de query, of hybrid search met keyword.
|
||||
```
|
||||
|
||||
### Sectie 4 — Eén observatie
|
||||
|
||||
Iets wat opviel:
|
||||
- Welke vragen werken het best?
|
||||
- Hoe lang duurt index van een PDF van X pagina's?
|
||||
- Verschil tussen vage en specifieke vragen?
|
||||
|
||||
### Vorm
|
||||
|
||||
- Max 500 woorden totaal
|
||||
- Concrete cijfers + voorbeelden
|
||||
- Mag wat informeel
|
||||
|
||||
---
|
||||
|
||||
## Bonus (optioneel)
|
||||
|
||||
### Bonus 1 — Streaming antwoorden
|
||||
|
||||
Gebruik `streamText` in plaats van `generateText` voor word-by-word output.
|
||||
|
||||
### Bonus 2 — Source-citations in UI
|
||||
|
||||
Toon naast elk antwoord de source-page-nummers van de top chunks.
|
||||
|
||||
### Bonus 3 — Multi-document
|
||||
|
||||
Index 2-3 verschillende PDFs. Documenteer hoe AI ermee omgaat.
|
||||
|
||||
---
|
||||
|
||||
## Inleveren
|
||||
|
||||
1. **GitHub repo URL** in Brightspace
|
||||
2. **`RAG.md`** in repo-root (4 secties)
|
||||
3. **Working app** lokaal demonstreerbaar (`pnpm dev`)
|
||||
|
||||
---
|
||||
|
||||
## Beoordeling
|
||||
|
||||
| Criterium | Punten |
|
||||
|-----------|--------|
|
||||
| A — UI + indexing + chat werkend | 5 |
|
||||
| B — RAG.md compleet (4 secties) | 3 |
|
||||
| Fail-case analyse is concreet | 1 |
|
||||
| Antwoorden zijn echt gebaseerd op PDF | 1 |
|
||||
| **Totaal** | **10** |
|
||||
|
||||
Voldoende = 6+. Bonus telt mee bij twijfelgevallen.
|
||||
|
||||
---
|
||||
|
||||
## Tijd-indicatie
|
||||
|
||||
| Onderdeel | Tijd |
|
||||
|-----------|------|
|
||||
| A — UI + chat | 45 min |
|
||||
| B — RAG.md schrijven (incl. testen) | 30 min |
|
||||
| **Totaal** | **~1,5 uur** |
|
||||
|
||||
---
|
||||
|
||||
## Veelvoorkomende valkuilen
|
||||
|
||||
| Probleem | Oplossing |
|
||||
|----------|-----------|
|
||||
| Scan-PDF — geen text | unpdf werkt niet op image-PDFs. Kies andere PDF |
|
||||
| Vector dimension mismatch | Index én query moeten zelfde model gebruiken |
|
||||
| Top-1 is altijd irrelevant | Chunks te klein, of vraag te abstract — experimenteer |
|
||||
| Cost-pieken bij grote PDF | Indexing per chunk-batch van 100 ipv all-at-once |
|
||||
| RAG hallucineert | Prompt strikter: "Als info niet in context staat, zeg dat" |
|
||||
|
||||
---
|
||||
|
||||
## Tips
|
||||
|
||||
- **Test met simpele vragen eerst** — zorg dat basis werkt voor complexer wordt
|
||||
- **Log similarity scores** — onder 0.4 is meestal niet relevant
|
||||
- **Eén goede fail-case is meer waard dan 10 succesvolle**
|
||||
- **Houd het simpel** — geen agents, geen multi-step. Just RAG.
|
||||
|
||||
Volgende les: MCP — Model Context Protocol. Tot dan!
|
||||
175
Les16-RAG-Embeddings/Les16-Huiswerk.pdf
Normal file
175
Les16-RAG-Embeddings/Les16-Huiswerk.pdf
Normal file
@@ -0,0 +1,175 @@
|
||||
%PDF-1.4
|
||||
%<25><><EFBFBD><EFBFBD> ReportLab Generated PDF document (opensource)
|
||||
1 0 obj
|
||||
<<
|
||||
/F1 2 0 R /F2 3 0 R /F3 6 0 R
|
||||
>>
|
||||
endobj
|
||||
2 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
3 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
4 0 obj
|
||||
<<
|
||||
/Contents 14 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
5 0 obj
|
||||
<<
|
||||
/Contents 15 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
6 0 obj
|
||||
<<
|
||||
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
7 0 obj
|
||||
<<
|
||||
/Contents 16 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
8 0 obj
|
||||
<<
|
||||
/Contents 17 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
9 0 obj
|
||||
<<
|
||||
/Contents 18 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
10 0 obj
|
||||
<<
|
||||
/Contents 19 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
11 0 obj
|
||||
<<
|
||||
/PageMode /UseNone /Pages 13 0 R /Type /Catalog
|
||||
>>
|
||||
endobj
|
||||
12 0 obj
|
||||
<<
|
||||
/Author (NOVI Hogeschool Utrecht) /CreationDate (D:20260607091455+00'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260607091455+00'00') /Producer (ReportLab PDF Library - \(opensource\))
|
||||
/Subject (\(unspecified\)) /Title (Les 15 Huiswerk) /Trapped /False
|
||||
>>
|
||||
endobj
|
||||
13 0 obj
|
||||
<<
|
||||
/Count 6 /Kids [ 4 0 R 5 0 R 7 0 R 8 0 R 9 0 R 10 0 R ] /Type /Pages
|
||||
>>
|
||||
endobj
|
||||
14 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2044
|
||||
>>
|
||||
stream
|
||||
Gb!;dbBDVu&Dcq.As6.NXUmVi!5!:ja76.X]]<V`%7NSrD2M:`b"f;,[s%gM+=16tX<qXuq,$&^Qj5<dkCO*jI[I[cF+EEuGkR:$j"Qo%F,::"1H6F9T,kds*k37&+fP`-OAP.*.&TfMSko9GLC>F;L^[O-,pQ<B&ie/E!Z"&#@.MfU5Q[P^dKLh_k=bj+HpN/*dBlP/h[_[/\,Jtd?:b"F4*7h)7qFZ3[6[rt_BkFlJHqr1%/tYGnJ#:E;cUfCU<)9*/lb$7iI^Y$N_Y@"<L$Um9&PE7<`u\pX1\EL7[GfTWbqG$]>>eqTY\5=1K=JIi70TbDH(o3k^=CEdl[]_;tLP9Ff!4CXf&m@,R?PaS$@%R*?XIh*B3JVAYtRJ9h78:oon8'qblqOhM9ag2iJ&04cej<S]iH*?m/WhN.oM)CWoC[VtW/l>%:T"4)M>\(-<5daRIoDDniXnPL,^X\Bfh"aT]]00is%M)MtNbm:,5[<Jj'8A*3ldP'XElSVQS\CWoCmm.SH=DC)[=7QY,[3d.XL,9C?XFb.)VHS!*E=k/o=1o9^.-/.j2[:!J6[Tn,-N$ZaN_$'u*,,ITX:LTHtX#s]A!<WtYJ:uRS2^BPZg&k<$S5*&oDdU/1?f:hr0COrsL:+2#1ku@/eL6@H./U&CpD+*>#W0t;U%ATBReL')#CL]nH+[;ooT'pJ\V7NCI#8i2.2Z+W^CD`\?of.<YcSOjOb8_BeSf4upUpngTCaW"X.-6W^)rWkbu#&CTHIR>.*9nBh&jb]^hop-W\0$t$b/et;1_@m^IbW\bH+J%71s0f]fQ;ToUp_0o*j6O<;HJ)m;$-?DbeaWeT-VF[j[](-M'XgF'03h2M`Fhc7$A"p5%e&5lToM>aDj`^Y:GO/'u2:a!K`p7Kmr;&?/\4DYO,(-RcV^LaHO?XrfQ0L0t?1mOi5*$a%:lcBGNMSdKTsc4hm\>XP$X$V_siHU,OCD0msemX%:'[g+P`mM<5(=p<B=m%&lL[o7m<_.q$J1![bAXOV!Q:cjAG9sFm;Hq`8qc'923A!k*1ViWu9<1_m&KnM:I'\<aJe3eU2@D`6%:>km%+6#uss*I*RmQ9HQ<&(hO?5`>MkI7c0KT/<0-?6CUGI-LU3d$GkSeoS>)QjYXC+'jZ.*,@f/<VbT(.R4!YKA\YlR7NKmP#F=_U,3Q.P.YhIF7`c/&\cm*b$'+UHX+@C*oti*oPN[hObP%ilEsC1p>H:ChC.n#*L0:C+Rn5Z@DGW*[Lh"[B7R>;Me[>PkPeNJb^FBlao?$)Q]=gVj/F35TW3sfopF/Nm^YQkC&2r)A>25L-LbP(kQ?g=DS&P\%7\BQnd0CPorO$=2-PHiD\;jUfJ)8<]%ec3q$eGA38J!d>PYK5+BcMR9W&g.;q#$O<Dm1>dI9OKLDE..E55'$OLU7H1SYGh_6:NX*667U^mp6>1;A?VkGd\")KTJ/4JgBchC9P&Q5)2%rRneUt@N5<JB\5b-7qrUiKK&B]0_&O^If%g)DsrKXSd7:$b$3f`_b8X\N>_2(U[KXIFX:QV"3sKt82^.;tAKp$uju0aTA%p`h!JCcs2@2[bl]X(#5O$>sHiXVT;NQ'd/9B@6%?#60X\g.93U8371Tr?FIemQO]DR9iXgb:"jF^8``QmKfbd]&,`FI;/@D5bHK*FJ4jS)Fe;/?E"@-bZ+1oYW=iArPL[\qF5N0o-ds8frdSh?lTD;lc!83&36dNb(mJS/Y()o<C<n&eAH9*iOS8iHC<Yf=BcXEp\JR<],09<IFJ$diu=M\mXaLOi.n$%,=$qR792guE)MpAR,bnS;WPr6^0#8;@Qb'+pg%rCe$:>5_R^4o=Xbpk>lRu@PV@l(^(\p:!E=HF9,/UeR3M1(DijiD.fWja`WP8K#7K=slQVAS[)abN#NUDFCfhVq#IF.&fgDu2^]".4fQ!E^]AA!YDr)Fs%t.gY.*AWB&('(8$S_R<3dP2;3b_oZ%PEFd(`A\CZ,1h88(mPtN/K!RHA`)?l".8J5^?lDB/!3CA$nXd/`d,\5Bm\,;Z~>endstream
|
||||
endobj
|
||||
15 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 437
|
||||
>>
|
||||
stream
|
||||
Gas2E?VeNm'ZJu(.F+]O=F0joNu0.((klMsj9r7SVQf#;p$5g?5`fYte!Kcuqn"uo*fAZmYm.?EoLB#s87"Q=.3uC48YYs8#pZP"@+'$4$a<`N(TK#9TFE]^L^B)Gk,a/T`12`+7*o(0(6up50-W)c\9//&OZ(LTkp8,PK[u#pA)cDB*M;*G`Q[]4!bSoo=a36ETan5'0i$G;,\pZ?h*4Xg/i-ifQ=[VD6W<g"ZGD_9(5'`ZTW[mtnL94le$$p=g)&![P#ipBD6G16,rJVagJe,V?\:2T+*mhHX1-Au3m[A+%WQ#9$_M`7C8G<5EtuOtf=\fbL2(5WD(OZXb$H1h\PrEidLeHog7n:2@W0r^I.9VUSe)r!8n+71d8)97q[[U+oc=gl='8S'09CYDWF+bjD^[Q3%S+V<o=Xe29$`sPg-YE@,,gtl~>endstream
|
||||
endobj
|
||||
16 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1620
|
||||
>>
|
||||
stream
|
||||
GauHJgMYe)&:N/3%/dJ-D477KOkLu_S2M/c5uT_3'IX_fA1b-g=?#PT`sM\4*E#s#[Y=>sX2R[migcFqS/De=MANV6;$uf/"\13_0Fk-Z^]Y7RH4A2%c@nWqc_^,]\;Iiu#84Q/"6J+U_"3C)n:6(aOshbE-q9F.'QDQcP"@&ZI!Xd++O[/c#p>+N^@K%fGDXX+RMNj5rSXi-kFXNt"iTlB6T<aQK&I'f"Z;saJl(T$%ot,1rd=>:"CC1o>t)@f:Z,Mal=a8:RWAeU-\o`B'j%g"^`WDN-[Cj>XPS`_B37;O[O1ddon[Rd]T)qp=26qCe\p"'0$dbW^VXiXL#>^V:2s3Z,bK:P8ud\_+J\SV*HtcD)<D34plIPl,OOM69,oPjV*[me>]m(M@FF=;M:\qfS3_l?9JL.P##Tp:!Y5KOQaLT5#C8[h%ee5*_sf5s?d\t2%TXd/Na="+B5k.T*l=:4*]Z=&P0.$HmZ1ZknlYV^7!8Epj=DF4"e7)NM!HNdfpk3pjFsg]S+#2]pqF]X#lbrQE8n,&M0fX1n)Kai[38bE6+B?M_0U+#b9De*H0bDh<X?NX<>e[gMXX_;_n@@O,mDj?iN;q*&Q6%`362_&-u'Gdd86`)Tn:Fd7.k=>B<5\UV,Ttne]W>4"S7Ra$]_T2(C_Do&qh2'(g]b0-ElY>7Rp[;)JY4^NkT]5@l36+^!q!EYaM9\1F(h:,LKbgR(LJpd1IYofo\uHWN5pN^nAcr3i?g2M"7XITa\DKF]_*pGR>etlc$=N%*".I=pTWfp8B`!l8[+U1O9c0N<Z+U"PET1fB7m+>MTub*c<fNnu+:f9P9G'E#)oPK?Gs$8uH)u`t;9j)CJr2f@OR%i,O*;.H2N@3dYjI2"fU:LIb)59(K=73iXo:\7fu0D+%B0`n>OK=RX/PH@HE1qSWb@+-$8oUScU8#cm8BJcXIG_:.:LAl"gJC`Nt3e)j,tB6pmMV_C'g<>]qC2jmu*/Gshof,7tW,6=XN"8iTd@3*50J6Om"Ltf%'=[_Sp?5hDBmR+?sZL_i"@+K[UCfP<D*noO:%/qp#:_'-#c^V"BJiC0bfR3kdNre%[a3:d^900X_lFJE#)tKLKoa(0&hV-55F+K7Wgll.&0l0@_-a6D,nqU<B!>.XIDF*r^+tr77CKMUt[PL.%9OGUCX7^V,(KNFu[[nba2X<\&o8QM2=CjN$A'F_VW"2=GR)W:A2bSVY&`8kRK;u(o(.jk^dFNB=-U$Pk'!82:^bI.9)m'e3)N2.SKb]$$kN0^`<-_D3\fr5/9Y2p>XuRJ+gSSI\9B>]s`o`H(nPCnA>OR!2(]WiD[_sP!r,,V^;QUaocL+#K^F/jO+-Fe7O3!6I,J;XI)8j6c-3efH/=snI^=@-tDj_TC8f[j8PLR>6%iRQPW(#dbr7n>'"R*LGGeSO?b8/RPjGPmZnj+@;dde7#<>/T`INJQ&\&$+<gSnbb2u42N=)0)204;e'SS]FTY/"%%0>u8h%u#8/^bN^4ZAHo4%Ht"L#5q)7SGp1b7U&(kh+7h>:6mf"_`59PUX0D]'mqk@k?/XfO+m<AU\eE7]uS-I>rsVNS.+>]1um7%pn?Gb5@&\Mq:&a/r!/>?I>%~>endstream
|
||||
endobj
|
||||
17 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1352
|
||||
>>
|
||||
stream
|
||||
Gb!SkhfIL2&:X@\Z#<.dTN\X;bC-pp9mZ%tVq:XPgcb/$Qj$amK3AK-rq3b+_)S,bg<4$0(DuM[ddFT:^>5&7\FebQDk$j@(rEU\*4UOX"LqtTL2Z!jk^7E<#Xg(h@%MDP8-"Q1qZl64ah\Ki&;c3qE^Q]j#uYc?<2;e+\hN1gPR*I#%L3aRDQ"g7`Vi%*Hg<g1#XPU>o7J0s^OCf2=a1Md2WtEYo-Im'K5Z?VXa6/@g^efMB$bo%qt4a7.]-n==])!aB2rD'X'd2@#_Ub-JME"\E"!Pm_R*EUQrdoSr!:1felTo+oR%uK2nWr+%3".g>V-Vq`cBqgIA`u7Y!I4>]dH^eQ3+<W)S1fOemt9kA_cg3)<Jt*#*:j^B?b9TQ9$4#Xt*Lb9W62ea`nYYmYJWK'0GRuh/L?D2:W1*1b\En[I*Kr:((Gk&/2kHIHkctb'+BPer\X4M&geq,,KcmOIO4+(nJ`XBBI^<ZCbatT=qPiWRVqA[F%KWcQ^AS,_Q4oG!4rfHI#o@p4`$kk53bjrLgS@0sPY($Kgr9KR:c=LV[m[)oJV`_K(bc,e)1S\"jQ`\!=h.Xpa7/)Wc*`]6FkHL>F!h<sd=/>)X7d]X4\n,bGcp@!$t,fb:li"2q;`Doj-]h3nc"(5hL*72D=<7teMs<ZP"HZS"GT&J)d3+A#Hp-NS@;61[6ISDQ4'!_oN_Z,sE!i$fB9(e`$*`Bf7d4S<qrm<j=`<!/[m3e^".FFGr7Je@G2"gdgn,Jh=)ANi<sas0CH6\2nW&bJfg"g-4O&u9h\BCG<bCs.AOb(hSSU+$=2Qt784=VYY.YU];[o7J%LnG$9Ohate0;3ZUqr4-sJVrp<l]V_rRhe9#`.pZ.H:;^):TR,L34_.ZG8[9VlAQ!p\',Mr5FokPR4>_+c_]4Gg:==Tm%A!R;%K6N>Q6o^L>ls[3Qk:&YTK-IAL7')=)!!aVCp9L/fM]0/'mqHr1c8QHIKE5!<"D]eChTi&:\tHC]>-24GG$u"8"QM0Y.i'1"]"\F-'IIC7cL/BNaSkb=#_L2UY^0MP4<phMf^`U6g_GL'?iX?:E5q1[TaBr[0.F+p>]UR*VASSOPYAeBdM2:X(0*?'[$6(/(8Db^DqM_c_7j'=Ah%'L\82>,^\g+gZJO1>-fet?r,m+YrQnnmJke1]@UC)#cX]^8k@C>2To,^LVMU+Ik87)T_i(k-7%q=->Ea5HCa>H2QQd;[GB6`dJ3_,7tPM-kq+_`D-pLDo<kusihB(1ano,@?]jNf,D%;Wn\u^+E`.PRJ@-u>bb!G#Fsq^!l!&H]GG/p"R%H`NjDg"k6C`Fn&:3Uu.scA+"`M'2B#+3I_4A=P~>endstream
|
||||
endobj
|
||||
18 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2116
|
||||
>>
|
||||
stream
|
||||
Gb"/'>>s99'Roe[3%s*]V(n8eg[a^Tc9PS@4*6h;d(\un*0IbPQE_H6p=Jbqlo6?Q#"s5F'&qQ+^2q#`F-?lSp`fDfaT)g*Y]iWg:`*L>@4m2k$%*RN2i^Nu)R09IT6l#\_2Wm7#NaU[+UHmXVD/,>74sW<0MYlu@h'4971kBLP]6<8+lGh`(NBX3-UGhk#P3@&U_H6R2[I<GEIOr12sSA"fP61V2K&`9&Yp`'U&*"+8.YqYb[BV;^V:i-OWM[qR:r_\]ImbNWSrHW#kj;aqc/n^[2d#0Gq5.417?at@JCjtDiCb=54\lCq-J.bs$TmQ!]ZXGeIPeh2n5bpSE9m)*meOVl3o,;Xeq6$?/Imofij:q5$E^</7]o7%Lq4UC\LL_1iba?n_Q(EgJQ0@;+UR[K%ZO0l(6iM_?hqnh4kh-`;t)55&&#Y$HsP:W8R$SEX;Y:k!+/4OOm&q*,i!6_OF<BG&g*Ja#n>gNcciCC=[k&co@*ZhU(AdJ+V;CFpX!SjZjC/k5i&-p:e!g/nrPBGjXIF(-d!,6[VAZJAgZ]p[A]85-<aP7i-<JF",YYlP6#<q.rUaSoE_7p?.hN8Ct0tcn7Fno/U4NfdU^lVl)0dg1Y/624-;_G:lE#h,CS^,VEUmE7m9T%fmW:3@rmaH'`$nq19b;Z8<Z@/gP-_N);&TD9d!r$(P2b*%F*''04-fa8eH))A4@EXBMSG7m"GT'kX`mBn`=ob;()(,t44qQF!c*-Ls2lcq^LMWlHFJQE-u_K3J8Uf<G#FbH[\]H6iAd=Umn^R@<aT=/i32\_IDPa"Suc1"&;W\*ZH-7I(V&Cd>\#Zb]>d,AhBH#GoMF0b@AGIka4P_&r/A:LkpK:/4!@47bR:8,WUZ5Df05WN+Ebq3R<>#6C@/5HN_YD(3C?W^>b]A5\^<1rTRA4tgU%PuOj.Y,sDX=4CSV;$2kBW2LeeECE%9't.bObV:sr8?L8ge1?X2<)ki#L.;tYmQX34&U.eO0s>F;kE<E$=UImk6[qeqUZj(Jab>K@ZP,lN3%38GUrKWARjQ$c@7I_6(-[`2g&=K#)]:tsgn<.8HhEp0CL^8?Z<HIfmZV/,6I+J?*.&U7o3ruh?:1QUHX_rEd&g$&WrZ?6YkM#\ZM_%j@p>%is,ENsMEOh4Hc\\*H_m=bElQO*`hfg!\?oQ"\t8%EfEPQaI2^dW\ho,rO9MJaKhsId)>:JjlcScrLiWBU`E4tC;G0V4mn0Y*bH(4bmQBXB&YfOf0%ulaX]?Yuo9"]FG>/KgR?L,>>c4ub!B;[lo+QWTA_lL`XX\K&A';m%YrNOa%L^nA/8U)"+iiOTJ0VW;QRsMk!LGEMq4PIYY<9KEa(cVk>OgR8.CQtT1g=r:iF*>=9&,CjSRP^o")>*r`G9#4fOF3bDqRZ\PM](N^5IP'P!@]<)oaY#o*of89a\:9rpk%Wc"g'L%Q\=1lt<X'&?dHb(]-*lCalYYUoQmmkA"kfQY<BmU9d0XbY0?FFZ!c`ZkdI(^EBhs+,X#,&lD43\PRKp9lut7G$&n"1rF7pP?%ApGa?QaCA/>Cq6N=VYoB`LFLWi4=_nYtJuf[7XPm"-R+XA_2+*5P)eK`*TD8F,RmG?!O/[b$SGDk@OCL%sZ]C+c^lL*,AFm;c[gKjNcBjoX>S^1`(pPoCOaqQVDOadpEf?#YMb>AD5)D9p8.0JIM6g#r'K0tpPff`Q%0N5TGOJ5Sn)]o:P+JhJDe@74"!ktM0Z<!m!Tk`Fq<"\\=S(A7k'7^r_/"QG]QR!?kuiEJR)$0iO?9X"/08asqo03W=:D>iq`L!erG6*6R`r-]:J=Gi0<%d1^TdS[RnD13.Fg"NWGA'XH?:[eoe]ZcmP7D=`=8X5WnrKXCfTFe,4YbP(n:(Sc&9u7bn`1t.0d+2+O..J0LM:Z2B^8oY@oRp_T9YhKj=spBpFN9R5A#(m9qF!9P&JSs%%9d>%S\Rl2o6BpZ2M\mVT^K_ci?=pF%5($Z9FjZ[uAt[?Ba&qPrB(iY-<gD1Y=D"hZ2D@@=:G[*:TCi/X_PrI52Yh,Ul,TI;O5eUW4SdNk*931/K/,*\NCBo^G/41Cc6P1/aMl-!,p$":PN^UOS/E;g$)SW?5~>endstream
|
||||
endobj
|
||||
19 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 774
|
||||
>>
|
||||
stream
|
||||
GarnT_/c#!&A@rkhPE#BAunNG&00.WR:IYn2H/=9EZq['8hdafG1qCn=5__k1GoU3_S3KT]2&cs0/Yqo-1IVDfqj:X:cq!WTi'V'U6tnJM("@B:*AOi`d%]La5I9TbW7#K8Fn62aJ9E9dEn9o=dE]V(h\d_E.j"qR=[,l1ms*><0CkX)gj3b.,rZY,\`\hj1"lj+>4Pq<ZTARDC4BP[GB>m2:b*o`$?as3;NL6a#1$rL4$Dmq\J&o8(D(a>ne0H?dI'3[sjtgT4gT/^KOMo]LVLJi\].Gi7DrmF=b[PrYtE)$*Mk15EgG_Ge]Q(m7O/Z7(:t$2bI!hM(FdXLV]V?h%R[[PkQbL"0)B%CKI3:N*8Q;FDA*Lb5q>^M*l77r&EMf$*&EHA/nGC0]C.;4cPWX^-DrR`eUtXdY@dBLQ`rk`G<=.EgnjJjf#Z*!^BK.c/)<!^Y+glAY/G+738]eEaC,_NV8HtRMW*t&5@V8@frX6O2dH[EMK(A?TP91GWU*A@"K"<%RX0icd+5sGUa:&hs<<Om)kB!%/'7W]tO5_)"(_Oadk0k"o/0l)W:"'7D/#I.i[b=jSWPjJ=:LsqBi#k[WH]f"(1_]rA8i!mqQm)D"Ar-A(dT_9^8!b;g-2Lhtja]BO9Ac*,OMXr3_S+RXSNa-%9+#];I[f?BMS1l`_ELEi[dZ:'HE#=DWo8o`K!.b79SdR_#EWZ57_)U-]m#GsJ3Ck>gaqE3a/4kH]4&M!#!<F]FR]oPI7)c:']1\)W1q57ok&8c~>endstream
|
||||
endobj
|
||||
xref
|
||||
0 20
|
||||
0000000000 65535 f
|
||||
0000000061 00000 n
|
||||
0000000112 00000 n
|
||||
0000000219 00000 n
|
||||
0000000331 00000 n
|
||||
0000000536 00000 n
|
||||
0000000741 00000 n
|
||||
0000000846 00000 n
|
||||
0000001051 00000 n
|
||||
0000001256 00000 n
|
||||
0000001461 00000 n
|
||||
0000001667 00000 n
|
||||
0000001737 00000 n
|
||||
0000002030 00000 n
|
||||
0000002121 00000 n
|
||||
0000004257 00000 n
|
||||
0000004785 00000 n
|
||||
0000006497 00000 n
|
||||
0000007941 00000 n
|
||||
0000010149 00000 n
|
||||
trailer
|
||||
<<
|
||||
/ID
|
||||
[<6fd13a5fbc4bdde06c9319c5650c0a39><6fd13a5fbc4bdde06c9319c5650c0a39>]
|
||||
% ReportLab generated PDF document -- digest (opensource)
|
||||
|
||||
/Info 12 0 R
|
||||
/Root 11 0 R
|
||||
/Size 20
|
||||
>>
|
||||
startxref
|
||||
11014
|
||||
%%EOF
|
||||
277
Les16-RAG-Embeddings/Les16-Lesopdracht.md
Normal file
277
Les16-RAG-Embeddings/Les16-Lesopdracht.md
Normal file
@@ -0,0 +1,277 @@
|
||||
# Les 15 — Lesopdracht
|
||||
## PDF Q&A app opzetten + eerste embeddings
|
||||
|
||||
**Vak:** AI-Assisted Development
|
||||
**Opleiding:** NOVI Hogeschool Utrecht
|
||||
**Duur:** 30 min in-class
|
||||
**Status:** Tijdens de les afronden — daarna huiswerk uitbreiden
|
||||
|
||||
---
|
||||
|
||||
## Doel
|
||||
|
||||
Aan het einde van deze opdracht heb je een werkende **PDF Q&A app**: je uploadt een PDF, hij wordt gechunked + embedded + opgeslagen in Supabase pgvector, en je kunt er vragen aan stellen die accuraat beantwoord worden op basis van de PDF.
|
||||
|
||||
---
|
||||
|
||||
## Vereisten
|
||||
|
||||
- Node 20+, pnpm, git
|
||||
- Supabase account
|
||||
- OpenAI API key
|
||||
|
||||
---
|
||||
|
||||
## Stap 1 — Nieuwe Next.js app
|
||||
|
||||
```bash
|
||||
pnpm create next-app@latest pdf-qa \
|
||||
--typescript --tailwind --app --no-src-dir --import-alias "@/*"
|
||||
cd pdf-qa
|
||||
pnpm add ai @ai-sdk/openai zod @supabase/supabase-js unpdf
|
||||
```
|
||||
|
||||
`.env.local`:
|
||||
|
||||
```
|
||||
OPENAI_API_KEY=sk-...
|
||||
NEXT_PUBLIC_SUPABASE_URL=https://...supabase.co
|
||||
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stap 2 — Supabase: pgvector + schema
|
||||
|
||||
In Supabase SQL Editor:
|
||||
|
||||
```sql
|
||||
-- 1. Extension activeren
|
||||
create extension if not exists vector;
|
||||
|
||||
-- 2. Chunks tabel
|
||||
create table chunks (
|
||||
id bigserial primary key,
|
||||
source text not null,
|
||||
page int,
|
||||
content text not null,
|
||||
embedding vector(1536),
|
||||
created_at timestamp default now()
|
||||
);
|
||||
|
||||
-- 3. HNSW index voor snelle search
|
||||
create index on chunks
|
||||
using hnsw (embedding vector_cosine_ops);
|
||||
|
||||
-- 4. RLS open voor demo
|
||||
alter table chunks enable row level security;
|
||||
create policy "demo open" on chunks
|
||||
for all to anon using (true) with check (true);
|
||||
|
||||
-- 5. Match function
|
||||
create or replace function match_chunks(
|
||||
query_embedding vector(1536),
|
||||
match_count int default 5
|
||||
)
|
||||
returns table (id bigint, content text, source text, similarity float)
|
||||
language sql stable as $$
|
||||
select chunks.id, chunks.content, chunks.source,
|
||||
1 - (chunks.embedding <=> query_embedding) as similarity
|
||||
from chunks
|
||||
order by chunks.embedding <=> query_embedding
|
||||
limit match_count;
|
||||
$$;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stap 3 — lib/embeddings.ts
|
||||
|
||||
```typescript
|
||||
import { embed, embedMany } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
export const embedModel = openai.textEmbeddingModel("text-embedding-3-small");
|
||||
|
||||
export async function embedOne(text: string) {
|
||||
const { embedding } = await embed({ model: embedModel, value: text });
|
||||
return embedding;
|
||||
}
|
||||
|
||||
export async function embedBatch(texts: string[]) {
|
||||
const { embeddings } = await embedMany({ model: embedModel, values: texts });
|
||||
return embeddings;
|
||||
}
|
||||
|
||||
export function chunkText(text: string, size = 500, overlap = 50): string[] {
|
||||
const chunks: string[] = [];
|
||||
let i = 0;
|
||||
while (i < text.length) {
|
||||
chunks.push(text.slice(i, i + size).trim());
|
||||
i += size - overlap;
|
||||
}
|
||||
return chunks.filter((c) => c.length > 0);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stap 4 — lib/supabase.ts
|
||||
|
||||
```typescript
|
||||
import { createClient } from "@supabase/supabase-js";
|
||||
|
||||
export const supabase = createClient(
|
||||
process.env.NEXT_PUBLIC_SUPABASE_URL!,
|
||||
process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY!
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stap 5 — Index endpoint
|
||||
|
||||
`app/api/index/route.ts`:
|
||||
|
||||
```typescript
|
||||
import { extractText } from "unpdf";
|
||||
import { embedBatch, chunkText } from "@/lib/embeddings";
|
||||
import { supabase } from "@/lib/supabase";
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const formData = await req.formData();
|
||||
const file = formData.get("file") as File;
|
||||
if (!file) return Response.json({ error: "No file" }, { status: 400 });
|
||||
|
||||
const buffer = new Uint8Array(await file.arrayBuffer());
|
||||
const { text } = await extractText(buffer, { mergePages: true });
|
||||
|
||||
const chunks = chunkText(text, 500, 50);
|
||||
const embeddings = await embedBatch(chunks);
|
||||
|
||||
const { error } = await supabase.from("chunks").insert(
|
||||
chunks.map((content, i) => ({
|
||||
source: file.name,
|
||||
content,
|
||||
embedding: embeddings[i],
|
||||
}))
|
||||
);
|
||||
|
||||
if (error) return Response.json({ error: error.message }, { status: 500 });
|
||||
return Response.json({ chunks: chunks.length, source: file.name });
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stap 6 — Ask endpoint
|
||||
|
||||
`app/api/ask/route.ts`:
|
||||
|
||||
```typescript
|
||||
import { embedOne } from "@/lib/embeddings";
|
||||
import { supabase } from "@/lib/supabase";
|
||||
import { generateText } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const { question } = await req.json();
|
||||
const embedding = await embedOne(question);
|
||||
|
||||
const { data: chunks } = await supabase.rpc("match_chunks", {
|
||||
query_embedding: embedding,
|
||||
match_count: 5,
|
||||
});
|
||||
|
||||
if (!chunks?.length) {
|
||||
return Response.json({ answer: "Geen relevante info gevonden." });
|
||||
}
|
||||
|
||||
const context = chunks
|
||||
.map((c: any, i: number) => `[Source ${i + 1}: ${c.source}]\n${c.content}`)
|
||||
.join("\n\n---\n\n");
|
||||
|
||||
const { text } = await generateText({
|
||||
model: openai("gpt-4o-mini"),
|
||||
prompt: `Beantwoord op basis van deze context. Als geen antwoord staat, zeg dat eerlijk.
|
||||
|
||||
CONTEXT:
|
||||
${context}
|
||||
|
||||
VRAAG: ${question}
|
||||
|
||||
ANTWOORD:`,
|
||||
});
|
||||
|
||||
return Response.json({ answer: text, sources: chunks });
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stap 7 — Test in terminal
|
||||
|
||||
`pnpm dev` in één terminal. In een ander:
|
||||
|
||||
```bash
|
||||
# Index een PDF (Tim deelt voorbeeld-PDF in de les)
|
||||
curl -X POST http://localhost:3000/api/index \
|
||||
-F "file=@./polderfest-lineup.pdf"
|
||||
# {"chunks": 30, "source": "polderfest-lineup.pdf"}
|
||||
|
||||
# Stel een vraag
|
||||
curl -X POST http://localhost:3000/api/ask \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"question": "Wie zijn de headliners?"}' \
|
||||
| jq
|
||||
```
|
||||
|
||||
Verwacht: JSON met `answer` (accuraat antwoord) en `sources` (welke chunks gebruikt).
|
||||
|
||||
---
|
||||
|
||||
## Eisen
|
||||
|
||||
- [ ] pgvector extension actief in Supabase
|
||||
- [ ] `chunks` tabel + HNSW index + match_chunks function
|
||||
- [ ] 1 PDF succesvol geïndexeerd
|
||||
- [ ] 3 vragen werken met correcte antwoorden
|
||||
- [ ] In Supabase Table Editor: chunks zichtbaar met embedding kolom
|
||||
|
||||
---
|
||||
|
||||
## Tijdsindeling (30 min)
|
||||
|
||||
| Stap | Tijd |
|
||||
|------|------|
|
||||
| 1 — Setup | 3 min |
|
||||
| 2 — Supabase schema | 5 min |
|
||||
| 3-4 — lib files | 5 min |
|
||||
| 5 — Index endpoint | 7 min |
|
||||
| 6 — Ask endpoint | 5 min |
|
||||
| 7 — Test | 5 min |
|
||||
|
||||
---
|
||||
|
||||
## Veelvoorkomende problemen
|
||||
|
||||
| Symptoom | Oplossing |
|
||||
|----------|-----------|
|
||||
| `extension "vector" does not exist` | Supabase: Database → Extensions → enable vector |
|
||||
| `dimension mismatch` | Embed model en `vector(N)` moeten matchen — gebruik altijd `text-embedding-3-small` (1536) |
|
||||
| `match_chunks does not exist` | SQL function niet aangemaakt — stap 2 stuk 5 |
|
||||
| `embedMany` slow | Normaal voor grote PDFs — geduld of split batches |
|
||||
| RLS error op insert | `create policy` niet uitgevoerd — stap 2 stuk 4 |
|
||||
| PDF parsing leeg | Scan-PDF (image-based)? unpdf werkt op text-PDFs |
|
||||
|
||||
---
|
||||
|
||||
## Klaar? Verder met huiswerk
|
||||
|
||||
Het huiswerk bouwt voort:
|
||||
|
||||
- Eigen PDF + UI + RAG-tool in een agent
|
||||
- `RAG.md` met analyse + fail case
|
||||
- Bonus: hybrid search of streaming antwoord
|
||||
|
||||
Zie `Les15-Huiswerk.md` of `Les15-Huiswerk.pdf`.
|
||||
175
Les16-RAG-Embeddings/Les16-Lesopdracht.pdf
Normal file
175
Les16-RAG-Embeddings/Les16-Lesopdracht.pdf
Normal file
@@ -0,0 +1,175 @@
|
||||
%PDF-1.4
|
||||
%<25><><EFBFBD><EFBFBD> ReportLab Generated PDF document (opensource)
|
||||
1 0 obj
|
||||
<<
|
||||
/F1 2 0 R /F2 3 0 R /F3 4 0 R
|
||||
>>
|
||||
endobj
|
||||
2 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
3 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
4 0 obj
|
||||
<<
|
||||
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
5 0 obj
|
||||
<<
|
||||
/Contents 14 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
6 0 obj
|
||||
<<
|
||||
/Contents 15 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
7 0 obj
|
||||
<<
|
||||
/Contents 16 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
8 0 obj
|
||||
<<
|
||||
/Contents 17 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
9 0 obj
|
||||
<<
|
||||
/Contents 18 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
10 0 obj
|
||||
<<
|
||||
/Contents 19 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
11 0 obj
|
||||
<<
|
||||
/PageMode /UseNone /Pages 13 0 R /Type /Catalog
|
||||
>>
|
||||
endobj
|
||||
12 0 obj
|
||||
<<
|
||||
/Author (NOVI Hogeschool Utrecht) /CreationDate (D:20260607091455+00'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260607091455+00'00') /Producer (ReportLab PDF Library - \(opensource\))
|
||||
/Subject (\(unspecified\)) /Title (Les 15 Lesopdracht) /Trapped /False
|
||||
>>
|
||||
endobj
|
||||
13 0 obj
|
||||
<<
|
||||
/Count 6 /Kids [ 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R ] /Type /Pages
|
||||
>>
|
||||
endobj
|
||||
14 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2150
|
||||
>>
|
||||
stream
|
||||
Gb"/'D,8qH&H88.0fl9K7p0=`M="9B`:cgl\3gWU97&$0TocdGP"e9+Kkl2:NK^H-L#o/d(#=YNBG3K$j-f\1B?&,FieKDKd1"t+'K4i(?lZ&9J9f(ckq6dJSI:3l_56;sE><Tu%O?#<+-`Zu#/23(huNpK,uN(m:GJSCTRc&b.@Gie_l%X4!Kht+jT/6&(J<JMmh3Gs0's6XLuA.-c9>g`$]+#J6SmJH"N(hQF3:5g*hgfE5"oUn2VIu3"2Erq\r+f(4GC\aeBqD.8"<3hmu57dPuP/OiJkS$BjT#*<rWTWcS/W`CCTbrmL^LbEZ'@.=294!94#PJBVG1<dj,<P%C"^JHX2..hAIhK7V(Gh\0#q/F@Y9%RH$lRIFAm.C]oDi@@dmN5U_unD#oc/)iV=\qE?]Vq`[;NaTA\[N!8Ss]9quBSAF78kn^TgZq$ni@Za%*N"j%=n_M1V+qL5##Ig-UV7;GtF/4`K&4b^6q!WUn/jd-I%=RE&:dof_qp'h3P[2N,(uK2JqJLXnO%0#crjD2!bU5SaMY2uQ&UbF&]I^qDI+i8dI#DbG'/2s.hF5Y=`>I]1]%dIG-;l=G+c@<%%]I#[#`Vff7Om1TT`YpHArK"9rASZRh=E@S70.](<S+Y(i!!e"@aot#S`i%.ZIA*q3'hY#Zj,Q$35A"s0;on1iR$`Wig$0R['?*)JQp?ri2Ur-]%$^4b6mg`3+C`[#L]R3g5#1qjc\"YWrf)\GlnSrB$8j$ldG?Ea:\7s-f3)1jo&LJ'`=C9/d?0Uc-:tjfdKJ1Wk-B+7lbNbL\8<+J.!t*9uIgF[sbJEXA+o)Z-FhT25D'4iMSHWYbR2A3"/GZpp56D?0UqXL'd=OR:</^qj?L[`qqVCpB*P0%_.onhuAS)=,es`R"6W:<YSApi*^\DGZ%_\$g63"`nU]>G`lf@OJmm^9a[)NNQgBi@:ckZ7cCso=_>2pqBhpH,aXoj[Uj!rlTn?T/NV`Y3#=5^LY.E.2h`0G#VSA*VVQdrRl5FZE!@E9+`j7>/AC'N$##t[^,$HI+*E_LD+<n,<=I(n@AB+8#AXNaL92X69)!j5]G&KCZT?*5Y1&+bAj,TDrsYHE6e4FFbKsB$NM\V_et0@;7UKN>]L_WJ2U=<)3h'dF7VRk#ijGe-F;+EI?s\L6@ATYW]u1_QQQ,]nVf5dnVo\6bd0hG-Ki'Z[_m"k/*G4tESsM0:i>3!I$UBfLK$,j#egBtW&Fb>s'$HkN6bq^`F'PC+/W3kEKlT?8$*4[l;ms0D&+_ViAa66\+'78HPHP*2p@sP#O^n:X#*Wt(mas-#3>\g,fU-HD1"LkD7t_(!mSeo";q=1/(5U[E9YoZ]f$iKsDR+Cpi-ABo>d"63_^MV-e;@d2;%XWlCPK@%_Q(RclX@=r=SnPj?O-@#SUnc;MIOB]Ub,H0n3?,7S$\L;P4&2c??3[XhiQ+5+QM<NRKoXR,:;NS3<cmqXiDKNh'Y5&$`>X9TZE#D#+R$LB6H.'^>IhA7j`P_.&[sXr/Ng,oi_8#S9^>PH\Kbb4##OKJTLeOiJQcdK7HJf&&EQi_;a?TYcthmn6o8b9bKV(dP0'ufAq!a1@BRG(@GZi@t)]WUgOnqZ(//P^*I^2r^^\I\anH/cQ$)eRFl`V2&;l,Q,=ZN(c+/M$:uKV^,J1Q$?Jt+c-J=#\7FD<COtk=W4l\b+dEs.VEGd`f.tiP]aqKZ[@?'#e).=(#+d95SYN6QYrHWI5&&<tB&AQQ6<*4]&"+;QfEc\:/&bOh6P4ed?%d0=[/,!<"e:KtkAoG=!K9M@QU4bNR2!&CKLpKT@-K[,d.j0nUT08BEDg6jR2("(Ml''J_,":)lTa4=(<8!]^O.nS7<)u"h/e=Km)Ij8NZh)(.niE4I(l#DP%!rt*lD%/.J2c^AqU;YB=Q1!'R"mtiGTK1U9dLUk#%N&IR:@k0uFs1rWg:!URj2p>!<3-?P1HNcN]:LFG$Xu[`8N/D:i7$4gYHhrE;u:/5,?dgY\T]GE+EB*WNDCf;n1a6jWSm`_NuBS0%,@^>@]AjnRbQJ,FV?Y^LooIeVAq>o2&7btWjq0TSWt_g/p.GWaZK1j8.;^G"?PRTGm:QjS9fjsNmImSbBChBqA(-X(THNR%/I>iGrIrr<UuFqa~>endstream
|
||||
endobj
|
||||
15 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1225
|
||||
>>
|
||||
stream
|
||||
GasaogQ(#H&:N/3%"`os"rg%E;b1+872n'Kco-F'-NaaY7,.!!U0"(pn_\P(h<\uo`AT91,0udI3AV;%(r,_8T2bkUYMnjR$C&AQ$WIZ<lVI$93dQYh;bRQ*KU\>V$qS7Q#F\ptK2b5b/>6H\9oKi1$min="rNr\RVHLeCgT[HLfUc=!f3OJ)>13)S1)D(Zli8GjQ62gT*b-&U'>d3Zi;0cm\#22'.OeDVG^&h=0VQ"O2!lqDf04BD!tU8?[Q@i]n[&,)W#sko=f("f%V0nZ!VA4%[F0p/Zeu,p$6#O$st\S=<dA`r[6G%Q%0ZpaI@:'r\N@FFUIIDlSHP'"B0+Z)c2Ml'_;nK)UehEo<f*X#[2mY-;<EF!`^MQmK5YTQ'!b?3&iQc=]$F[Ys)MLj.7fjei;2JF2h!Pdoo'N_'P&uSC)f7`*cb7W-M913&'L]G!(DrZ`Pd9B/1c.-Eb8^fpd8S7Jl;EY41gInFcs>jJ:e&QpE*4\O48n1'K4k6qQ75/o,:g/_G!uensA\.(2r6NshnB(;AMH]B7q!MdZZDS7l=4hN^eR_^X5l<M)I>(:&?i5IRe!/Qe,piqda<TZH)cfX8.:dLm$a;5MML*prK8!CRb;F".EZeoa]A4OF%jf6(/\GAp)s(5gg!@<"`Q[>2n0N?RpIYSId:26hC:A#fj(k=*''_S0/MajoNs$&(,i]@;To^JBta;;'NueH'^&9tN1MHJY_][gRG#9LPD,'B&"r+o0V),-aJ<B:ip-A7EK&p5:u@hZ+r]#bYsclhe8A7P293`I6n<[_6m?G+"L+VoEa4QBpAcQV+UWC"Be,-ptn)en/2dpu<!iA6=oh_MNmqMsL_QEa(PC#@"s34^()a=J^a!6f,8SJ[?uE4%O.?kbGCcq51Kf>Bhdu*arpZ33=d;AVMI6M1rlU'8gq&SGRt5)LEl2XGh!tB0N\V.eVKo^@uEB;J@Zg<Dj6s4a"1nhZ'dh7qgL_ojH<E@[5QI,TL%I-n1*O@sutYm?$VXWp^LmN%\r;m_%b';H-)QL?-7\FK&Us%X][D.I\W+==r!Jf7dha5,!dY_eBK;2qKc`6A.%IND7K[(m.*?Ur`#EiBVaYpV[lZJ^bt7UB<L6XZ%)V!Fj?k%mU'c1,Bb8A3WTF\flW#&['A,ho8[R-%!qT6][5i>a&ZC4riO.#VPd9@`"V`J6\fjr`i\9oeKjkl'n/d([imdr<'#$QXg~>endstream
|
||||
endobj
|
||||
16 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1376
|
||||
>>
|
||||
stream
|
||||
Gb"/&9lo&I&A@[:m._j(&rc8i\'?gu-<&6d9f#drM/luPfO@41MF]g(4oXnUO;()CNgWg@Pr&]U`4k&UqZ+tO."5=r2d6AHT>ScD#)pIt!gk5Gk-*^^4Sb?UY.9Z%Lp0i<%`F4nJaI#HK$m%KmMK&@Lc#g@A].bbJYShlbeVfI`KPrU0q'"2JQSeVeR/(ZU,g9@OsmHgIK)Jq)q&dqpXruDh%.>@#4%2)W'V->PYj,A_rURDJ"OW^eYTIU<E0e'=0qSQWdt?:#+hC3T%.si2gu(JphO59S$ao=A!m8uo-Y^4!+0b<\<H%ds-9d=.DVY2/@T>HrZf,#b7qkplOEEr"4OnYekf"mTEb;`.-ZRREib@+C02igOmY!YMM)p8WO)aH\,@jLR*g:p;7ehk&oDm0/7S]1RN0Uo$6lodh40kJcB)Id]B^YLplt.fdKi6NPUrO*/.0%H8bUH7N'ZQ.Z:\;+Kl>TBPF"R"Xai:6]gN'AA!CrU\,_8n_CF<K9b."d=@_u]EIb5sHq3suplZ<0*,>3Js3E%J@,e:&MZ!F0?bc?YAR8]oUoDBk+WMt+nQ$9]+ula#3-N_`-%QF7kI&.XM);)6K@*^FdaI'`2oUKN$'j!K/Ts'#9;5-Wlk%dWKo.c]W5()M[\0dm',-0;7,$N$;&F'RO(lP,qaa!gG$b:+BA0udCY<&Y-jp'Ym2CCD'ok9!GgO)G7Wpr<f:-O0^b)q<RWcSeC&&Y7i*oomi@)s3IXHB]Lj0,<P$B"/<Z>DCH%buRc"H_?_b"J^KiE:C9L*u9Opk$i0!;s'j(;:o2glFkdMHlR-UJ`$e%F<^Z@00K*!P@Rl?"@bof=[^UW)b*IR95P5+Z-64?5RG0<II*)-gF3=d)ouQg!b`0.%j+;lH0;k-6n#gQD8iP?m:s(Z"g>'S<EY62E%THR]ep>?HZD:C#BCiArQnr$d<GE&V)ZV*n3/qk.0N>]%Z4bRnc*/Q'$:>OT1d\OI7oXF-OEVIdA`N7%MbD6d/3E4ocVmCR@uRGtaQN;OU9omN.DZ?_Dn\tck9#-OB4'l9<8$J4s<4!$B,B_bA51Hah/FX(GW?H':76a@r]g$5U80ZVTOANYZ-s48JcMUEqQf7$r^!>04ia`dg\\R#?"r&2inLPYWsh$=b*KaT3sAF"aQha!3NFMpm8,,M][GPebi$P/.e%R7?a,_[m)S(2a(b)d1*of1=M5W2\dY=VV.H%m8'5/kHhBIR.*'<H/dQn47fPp-sj_Us[*(hXlXZQ8-P#Qdd(_>+RM6nY!`1Wu.QAbK,0j@/+<R.hjnC<iD;ee83_JW?&%)t/^YiV*Mp6JCH!GKB7L%]R1_4/D6"+j\?.^B%[o2cY=r&'C@EP5bO%8O-;~>endstream
|
||||
endobj
|
||||
17 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1166
|
||||
>>
|
||||
stream
|
||||
Gasao;3RJ-&:WeDm+ddXj/s[O,iMKZ]pnYkRE7:"iq0)>J]"cfC;:&u\]i=spF]jh^3320#JjU@mC0-7-$*I9nb/Pb!4OI\gi;6R<ZBs0-*5t>0.ZM-6_T1!!MF@u*t#g1Da%!8]c7*-.MO$2__`8`5oPd'KE_STC!AN"(,i?Y%]C,k7KOJL.&6e'n"UMI6;%`O$=EWOq3ctDM!E'ABQd_aUD`use:?gWA3A3>X>61ApAU,^n8#;:<O+-44a_FL9m.!j+Z.uerSaWRd`U6#]Wki2^,TX]GLs(q!ioBsX:ot1,`5F`F8U6j!Zh=N`F3&l"8%@nS)uHH'<L)jY!&-b8$Z#I$k,-AR>7=&:f5VpYq(9%1lnYq+s;4<R4#E1$9m,9Si!Ng*)4>N-tJM*;iBo',i6%S7^RHK:kObS7-iNW$_t:)","EV:ufaUj/@qtq3d4MClFX(Ca??bQ\"JMJ';MP9Y?qJWiNZk&M3?t#[8$"1e4+,XBgXUi'Mu#kfMHDat4A?*tF5(O=nt/D''R*;*0]=qX<Mf+J_<co5=6MlqioVT"mNN>(!fR&Y<E0()^B-q-gtIqJcS^=4iJpB\C()ho7;%botcjq;5hp5;2`prM#jh:@i<[)57_G?nW?uoc_&1/,KCV2c#$GJ]B6FR)Bh6(<SP@I;C.Keb;1W9q<Xa/TUXUrmN?cQ1eVm+72F4.i2L/e)f6;(_!?!>$-<Na=@LbU<MK4/r%shIu@5q>K?p?`b6,n:F*ld_B:]+*&QJU!pj?7CL&d/C91K)1ToYr*iFORCjZ*hpqGU$Ce<!ar<AnCgmOY;N5%lhNdDE?j2Q,o:2K]$B+P@6)1A*m'=^e!I9B=0)U1;]F./BIEbfC<ZIKRm=d(D6j%8Mn_o'Y9i5/m#eN'[2e.lsAe/cdPSoY/SCMqV;\ONILGu_nV(t0L/.@PBVVkI7D>Qs\=?D10LdAWBa>2MKqZkOscMb%$f1p:^e=^DS4qlV#33bLNN3#2k-pcoIZ;3m6fV`CQK^=PS81n3\C'r$r4L]J$Z.dAdob>)MlSCoppYa;TrN=@aoJi2&)GuFK'e%[5kA:VhSAcR-p`-p=XpJLfSoKJIS.-LYHr^n0nQe'\u:-2-l+JDp8;AD2,mo\FX?fo&H[r1q)Uu.4BqX`AP&)KNU<E5X~>endstream
|
||||
endobj
|
||||
18 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2014
|
||||
>>
|
||||
stream
|
||||
Gb"/'>Ar7S'RnZ;3,f8TBk*U-/u1V:gNj00T]ftWg&^fl1,9LnOX3GZdnAj'^8&<1UfKNahFh?'P)dWPgck9KRK:A</,jlgJB(S*+#%(BVT6J'@_q[_4I[8P#2p%.Q4:cse9!\^f?F7;3AEk)*TFTaAK[rp;HR<u7\k,PE1A9Y'9Hle^2G\J_P';"5U<ZVl!0]r&r1O-D%YNBgoL#sqdaT99:X-c1pd*((li!h;[oa@&^"ou_fQ4&=0LMpJEP=::;'(1"n6l)BQ'$LN,o$7^)6g;VLFuu3BYh.3f)dl%^S<cT#1,KY,d1nEJnef&tAYLFUH=s]L;9P;%<.:FlJ/`ljhTNN,&mbnB+j!,r&+D0HXMODO;7\Bj-PY^s_.!.;p/.?uSC[[:DeIAMqUS9K9[8Z0O-9dE(g38]GeL7c`Pgf7jLaAEB2uJ6Lm)Jb=G!0RV'tQa?dfN>Bj5XVCk#1e(N0)`KgLHcEoNGa3d$oT'N'rY,KYLo`#Q8K4FX9?'\6LhfUe(27hNGp06to@7U]h^S&#_QW!(SCl%SmV^r)Ah""s@Te!]3h46Q^Hnt@l!W0k&HW'"dt(@!`\l7N93N$O^dUb<WH`,Z6Bpf>OTgIcBg/d:22fs-e=rZfOnBR;BU?$d+Pfn#d$71EQKARh"QqZp#;m8R&qk!AE^g<`40N?+7U'.Y.:h`HD/HE;\)+L$2j2#\\&]H#M%_L]3TEZ8HqAB_Y2o*i<#gs!m#P.m2GTuHVFd*1]GWO3N"_D?!;^aV,oNZR9Ba4^.eGJ\$K97q_s)!)kjo,YVU&=)c";<Afn2mGZ)DFQYQ.uDVAWW*0g!E\_a@7mPb6nuGtq,`.%C`ZHTu?PAJ4$_"PNk+b9K0Bqr(,!=@cbRH8_kTocjj"fWHdLh(m-X$Beb9C9B<L!H54McdIh`kpf_(.b,f$^"IA/<115j:lZI\pVi3H*9e[4pd#t3jNE=ufh(N(5#fnV.a05In+Rr/81;Sp-eA7$_>VXo:1%fSDQn_1`WlHUqK%B7.K(Sd'k<hj*YR\qUE6hm-HraL#^lRAm\NG%BU*ggrRYj$k)Y4N0O'%^%d'4ie>iEjfPHV=ia.@B5oBN[7\fNdcl3GL85.dT]le*I*1'cnPl(keMI<e"3RONq<2QQASdg#`@1/7aAPG:Qd9HniLghAS+'gm`Z]Pu^Q@A%m)a,aVCuUgK2ILM;/Rr>;a2_A;fr$5jk(jGe?U'5P("T@SP]8>uen*4b,t]=(mp=SgD>g_0]87iNjBUQ1K,hAh]W\=^,W]-U+mqg6N02Z@rc&5o9L%cP;T*i&8ue]Sdd0t4?ZRgbhEm"Z1+1V5^1X?lO4ZnYj-4u<'_r/dLjVg6=fJ'T&fgXHC'jn>pHB"&EZOFeHQV$;0i:V&]ao1SE_p73,KY;)k+V$f)@:8sBWGph^!B.o]7\3TP6C7GokYJ#nIk!\U1[)r\D=H"FmS)E;4G3u'#<^lU?@+sQGo'X3/B[3/>=tjR$?$n^%9.PrX?-dm;!I)R\Ks%*;b;9p^7t[;)$VD1%KZ*pn;$5,b?(g[_lqr[6$'g"IFb@T:(4$f=.qj\#W02N!6Z$]ofg&E(8/^n>,Cj7aj#qT&b<@i-QrW?K8\e%!j)#Pbc\HmTg=f$A\^#]_=:2ohRrN(G)q^kAa8FT*O\/^"D.%"Rn@?T(hV7bf0'liVY)n4"Wqm5;#FOr)AI/aF3Yq&Yj42cOF'jY*J-f$`1N3@B?%R!KS&9PV%RQ=GWZqD(6MS0oI..L,10pL5B1\0m,Cl7!3T;q>CLWgB<R50!ps6$"-9$__aS=.Qj!]j1Hob<[XteC6od5h7;2'qCWSbmPIj43e.J)r70bQ./3H@coj3D#P'J7RW<W?f"/!dQ8#G9SUeM;_4.>-GW>E$28=qK49QD!L,7:2Pn%lm[8,jgr8_sDI^Zi%/$1.',RR=lJ1$L9]I\2S*_m?DZ\J=8V7.b7n'K+)jl2JeQX>/ZOW>m9FOqd3ePW!ug"aWuHd.)M`<V?O!.8L#rr~>endstream
|
||||
endobj
|
||||
19 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1371
|
||||
>>
|
||||
stream
|
||||
Gat=*D/\/e&H8h>EF6:()9e6X.?PIO,"%MO?-kF5RfPm,6'3!gNSZ/,C%Le;Uh!VOS/@`&,a,`cmZ)\i:^eb$RCgI!^`P@WGK,/QR,%iX7)8-j:1rF-O77[T63?ZloecuKIJ\SAmM-iH&nnEc-lN^5$4r(,V6g;aIUU;aW"%N(Gmm4\cqk#kMTJ*echr2(E/pUqO[5YLrVQMW9i$tGcOSFjIL(ZDX=n<Ro`_$_L(\)Y*.5K53JW[WgbePq/2R(Teg=_1E6EM4a6/ZHVFd9.,]I369ib<*.G@.VI=p48NA\Qh>S+&,\;>]D)IO3I<`Vd4%FAIVH!3TIN3KI(?<:M8mun3[,/-i?>`Vr6h=GW,PV3(M5jiV`]&k]4Gs!@:$8ia^Wj`UoK`N9VaH]Kd%:e$uMD5Jr$u*)t&dE0EFZ)SgX+;*$>dZAKiD[168-^7u7#=(<c6C0OOC[fBPVK1#p(j7[Q2PmLs6e,%`t:9RWi*pOfbc#?1%#-2>[L(aQ;*FkKs5H`\*kR>^u9(GTAp;S,h'ij%K%/TZH'8P+%deX@\BaE]?F[["[3@UQ%9eIY\+nQ]5Q8IDo0JtmJ?'e2Os3@+kl?S/9^@9Hg+okE6/%aY#'Ibq<Ko?Q@ugqfs)Yj+Pn]eoI6utZHRO]B4MrID$gO>b:5#>%>su&SiYn3=9eVeW3fJ+o<U;CPElBbI5GqJeo)]\;'h+*"nNDHAm[S^c?*2XG>Vl.,\js2jEERhJN(#MM?m@m)_V31*);[ZI6Mr1jA?jLp*A#^S`Y,EMg:_X$j/dsbO*./gHur;Qj,OS%8(QWQ_D<:<r^*rAb".F\!cF\VH4RQ^F\Bqeki0>Kd\G5-BFmV(4OAhW*0$&9cI*j^Q](C)2iTb0a1M-AIW2SC!i;o?OFL,AVK5HAX:>3R/O\@@[XZGo2qGBGq#h<c<j>lTSdL-/BEd<mg7Z-:#i1>=O5`S1lpd2@Mt9_*kmD@;'ku+_phdOQI-o6XC3o:[CYo8s5-Z-Pi=k6#fflrGNZ4t8V-1<*g,r%:l'M4kA3)0.E^,imNY@s::00-9%7_fUNA3bm_-.eGSEJ[WJ[0*DB[<Rb.\(1Mi2_i\2L>mnX,ZW!kRnsiK)JZ6G2aEBq10/)Tre0&BWPffT+>qDcr-W9tGA=(+I7m`5f5DkcWFs)8cOT3eG4qK6$sm(8@d/']2KdBSf4[d@FP:MC-A9M4IDh[\[P.dFRoHV]1Z-\K(,Cs)*0;pX81LO]d4$Nsn#C1Q/"(4kq/k9as+'Gd0p]9-m(LRt^&[qB%b$9!-Q(3)/SG'eD(hKs0Za\N^!HC$hB)XtfLSBY-!^-P."ip='o\J#C<#+Yc"QfZ,5X@Bskg=seBV"7KTsLk69~>endstream
|
||||
endobj
|
||||
xref
|
||||
0 20
|
||||
0000000000 65535 f
|
||||
0000000061 00000 n
|
||||
0000000112 00000 n
|
||||
0000000219 00000 n
|
||||
0000000331 00000 n
|
||||
0000000436 00000 n
|
||||
0000000641 00000 n
|
||||
0000000846 00000 n
|
||||
0000001051 00000 n
|
||||
0000001256 00000 n
|
||||
0000001461 00000 n
|
||||
0000001667 00000 n
|
||||
0000001737 00000 n
|
||||
0000002033 00000 n
|
||||
0000002124 00000 n
|
||||
0000004366 00000 n
|
||||
0000005683 00000 n
|
||||
0000007151 00000 n
|
||||
0000008409 00000 n
|
||||
0000010515 00000 n
|
||||
trailer
|
||||
<<
|
||||
/ID
|
||||
[<92312eaeb815bcbbf7798059bade2d38><92312eaeb815bcbbf7798059bade2d38>]
|
||||
% ReportLab generated PDF document -- digest (opensource)
|
||||
|
||||
/Info 12 0 R
|
||||
/Root 11 0 R
|
||||
/Size 20
|
||||
>>
|
||||
startxref
|
||||
11978
|
||||
%%EOF
|
||||
465
Les16-RAG-Embeddings/Les16-Lesstof.md
Normal file
465
Les16-RAG-Embeddings/Les16-Lesstof.md
Normal file
@@ -0,0 +1,465 @@
|
||||
# Les 15 — Lesstof
|
||||
## RAG + Embeddings — AI laten antwoorden op basis van eigen documenten
|
||||
|
||||
**Vak:** AI-Assisted Development
|
||||
**Opleiding:** NOVI Hogeschool Utrecht
|
||||
**Vorige les:** Les 13 — Agents + Cursor + Vercel
|
||||
**Volgende les:** Les 16 — MCP servers
|
||||
|
||||
---
|
||||
|
||||
## Inhoud
|
||||
|
||||
1. [Het probleem dat RAG oplost](#1-het-probleem-dat-rag-oplost)
|
||||
2. [Wat is een embedding?](#2-wat-is-een-embedding)
|
||||
3. [Vector similarity](#3-vector-similarity)
|
||||
4. [RAG pipeline](#4-rag-pipeline)
|
||||
5. [pgvector in Supabase](#5-pgvector-in-supabase)
|
||||
6. [Chunking strategieën](#6-chunking-strategieën)
|
||||
7. [Index pipeline in code](#7-index-pipeline-in-code)
|
||||
8. [Query pipeline in code](#8-query-pipeline-in-code)
|
||||
9. [Wanneer wel/niet RAG](#9-wanneer-welniet-rag)
|
||||
|
||||
---
|
||||
|
||||
## 1. Het probleem dat RAG oplost
|
||||
|
||||
AI-modellen weten heel veel — maar niet wat in **jouw** documenten staat. Je interne kennisbank, klantcontracten, productdocumentatie, dat handboek dat vandaag binnenkwam.
|
||||
|
||||
Twee naïeve aanpakken die niet werken:
|
||||
|
||||
- **Hele document meesturen in context** — werkt voor 5 pagina's, niet voor 500. Te duur, te traag, past niet in context-window.
|
||||
- **AI zelf laten zoeken op het web** — werkt voor publieke info, niet voor jouw private data.
|
||||
|
||||
**RAG (Retrieval-Augmented Generation)** is de oplossing. Drie woorden, simpele kern:
|
||||
|
||||
> Haal de **relevante stukjes** uit je documenten op, geef die aan AI als context, AI antwoordt.
|
||||
|
||||
Het magische zit in 'relevant'. Niet keyword matching — semantic search. AI vindt stukjes die *over hetzelfde gaan*, ook al gebruik je andere woorden.
|
||||
|
||||
---
|
||||
|
||||
## 2. Wat is een embedding?
|
||||
|
||||
Een embedding is een **vector** — een array van ~1536 nummers tussen -1 en 1. Je stopt tekst erin, krijgt vector terug.
|
||||
|
||||
```
|
||||
"De kat zit op de mat" → [0.12, -0.45, 0.88, ...]
|
||||
"Een poes ligt op het tapijt" → [0.14, -0.41, 0.85, ...]
|
||||
"Voetbal in Nederland" → [-0.73, 0.21, -0.32, ...]
|
||||
```
|
||||
|
||||
Magisch: de eerste twee vectors zijn **dichtbij elkaar in de ruimte**. De derde is ver weg. Niet omdat woorden overlappen — omdat **betekenis** vergelijkbaar is.
|
||||
|
||||
### Embedding-modellen
|
||||
|
||||
| Model | Dimensies | Snelheid | Cost | Wanneer |
|
||||
|-------|-----------|----------|------|---------|
|
||||
| `text-embedding-3-small` | 1536 | Snel | $0.02/1M tokens | Default |
|
||||
| `text-embedding-3-large` | 3072 | Trager | $0.13/1M tokens | Betere accuracy |
|
||||
| `nomic-embed-text` | 768 | Snel | Gratis (local) | Privacy, on-device |
|
||||
| `mxbai-embed-large` | 1024 | Middel | Gratis (local) | Open-source alt |
|
||||
|
||||
Voor dit vak: `text-embedding-3-small`. Goed genoeg voor de meeste apps.
|
||||
|
||||
### In AI SDK
|
||||
|
||||
```typescript
|
||||
import { embed, embedMany } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
const model = openai.textEmbeddingModel("text-embedding-3-small");
|
||||
|
||||
// Eén embedding
|
||||
const { embedding } = await embed({
|
||||
model,
|
||||
value: "De kat zit op de mat",
|
||||
});
|
||||
|
||||
// Veel tegelijk (sneller)
|
||||
const { embeddings } = await embedMany({
|
||||
model,
|
||||
values: ["chunk 1", "chunk 2", "chunk 3"],
|
||||
});
|
||||
```
|
||||
|
||||
`embedMany` is veel sneller dan een loop met `embed` — minder roundtrips.
|
||||
|
||||
---
|
||||
|
||||
## 3. Vector similarity
|
||||
|
||||
'Dichtbij elkaar' kun je op drie manieren meten:
|
||||
|
||||
| Metric | Wat | Wanneer |
|
||||
|--------|-----|---------|
|
||||
| **Cosine similarity** | Hoek tussen vectors (-1 tot 1) | Default — werkt op text |
|
||||
| **Dot product** | Sum van producten (na normalize) | Sneller — als al genormaliseerd |
|
||||
| **Euclidean** | Afstand in ruimte | Zelden voor text |
|
||||
|
||||
Cosine is de standaard voor text. OpenAI-embeddings zijn al genormaliseerd, dus cosine = dot product (mathematisch equivalent).
|
||||
|
||||
### Cosine similarity waarden
|
||||
|
||||
- `1.0` — exact gelijke betekenis
|
||||
- `0.8-0.95` — sterk gerelateerd
|
||||
- `0.5-0.8` — zwak gerelateerd
|
||||
- `< 0.5` — meestal niet relevant
|
||||
- `0.0` — totaal ongerelateerd
|
||||
- `-1.0` — tegenovergesteld (zelden in praktijk)
|
||||
|
||||
### In pgvector
|
||||
|
||||
```sql
|
||||
-- < => > = cosine distance (1 - cosine similarity)
|
||||
SELECT content, 1 - (embedding <=> '[0.12, ...]'::vector) as similarity
|
||||
FROM chunks
|
||||
ORDER BY embedding <=> '[0.12, ...]'::vector
|
||||
LIMIT 5;
|
||||
```
|
||||
|
||||
`<=>` is cosine distance — kleinste eerst betekent meest similar eerst.
|
||||
|
||||
Andere operators in pgvector:
|
||||
- `<->` — Euclidean
|
||||
- `<#>` — negative dot product (sneller als al genormaliseerd)
|
||||
|
||||
---
|
||||
|
||||
## 4. RAG pipeline
|
||||
|
||||
RAG splits in twee pipelines: **index time** (eenmalig per document) en **query time** (per vraag).
|
||||
|
||||
### Index time
|
||||
|
||||
```
|
||||
PDF / docs
|
||||
↓
|
||||
Parse + chunk (~500 tokens per chunk)
|
||||
↓
|
||||
Embed elke chunk
|
||||
↓
|
||||
Store: chunk_text + embedding in pgvector
|
||||
```
|
||||
|
||||
Doe je één keer per document, of bij elke document-update.
|
||||
|
||||
### Query time
|
||||
|
||||
```
|
||||
User vraag
|
||||
↓
|
||||
Embed vraag (zelfde model als index!)
|
||||
↓
|
||||
Cosine similarity search → top-k chunks (k=3-10)
|
||||
↓
|
||||
Geef chunks als context aan LLM
|
||||
↓
|
||||
LLM antwoordt op basis van context
|
||||
```
|
||||
|
||||
**Belangrijkste inzicht:** LLM ziet nooit de hele DB. Alleen ~5 relevante chunks per vraag.
|
||||
|
||||
### Context prompt template
|
||||
|
||||
```typescript
|
||||
const prompt = `Beantwoord de vraag op basis van deze context.
|
||||
Als de context geen antwoord bevat, zeg dat eerlijk.
|
||||
|
||||
CONTEXT:
|
||||
${chunks.map((c) => c.content).join("\n\n---\n\n")}
|
||||
|
||||
VRAAG: ${question}
|
||||
|
||||
ANTWOORD:`;
|
||||
```
|
||||
|
||||
Drie ingrediënten: instructie, context, vraag. Variëren werkt, maar deze structuur is robuust.
|
||||
|
||||
---
|
||||
|
||||
## 5. pgvector in Supabase
|
||||
|
||||
Postgres extension die `vector` datatype toevoegt. In Supabase: één click activeren, schaalbaar tot miljoenen rows.
|
||||
|
||||
### Activeren
|
||||
|
||||
```sql
|
||||
create extension if not exists vector;
|
||||
```
|
||||
|
||||
Of via Supabase Dashboard → Database → Extensions → enable `vector`.
|
||||
|
||||
### Schema
|
||||
|
||||
```sql
|
||||
create table chunks (
|
||||
id bigserial primary key,
|
||||
source text not null, -- filename
|
||||
page int, -- voor PDF page reference
|
||||
content text not null, -- de chunk-tekst
|
||||
embedding vector(1536), -- de vector
|
||||
created_at timestamp default now()
|
||||
);
|
||||
```
|
||||
|
||||
`vector(1536)` — dimensie moet matchen met embedding model. `text-embedding-3-small` = 1536.
|
||||
|
||||
### Index voor snelle search
|
||||
|
||||
```sql
|
||||
create index on chunks
|
||||
using hnsw (embedding vector_cosine_ops);
|
||||
```
|
||||
|
||||
**HNSW** = Hierarchical Navigable Small Worlds. Approximate nearest neighbor. Tot 100x sneller dan exact search bij grote tabellen.
|
||||
|
||||
Trade-off: HNSW is *approximate* — niet 100% accurate maar 99%+. Voor RAG: prima.
|
||||
|
||||
Voor kleinere tabellen (<10k rows) kun je het index weglaten — exact search is dan al snel.
|
||||
|
||||
### Match function (Postgres RPC)
|
||||
|
||||
Best practice: definieer een SQL function die je vanuit JS aanroept:
|
||||
|
||||
```sql
|
||||
create or replace function match_chunks(
|
||||
query_embedding vector(1536),
|
||||
match_count int default 5,
|
||||
filter_source text default null
|
||||
)
|
||||
returns table (
|
||||
id bigint,
|
||||
content text,
|
||||
source text,
|
||||
page int,
|
||||
similarity float
|
||||
)
|
||||
language sql stable as $$
|
||||
select
|
||||
chunks.id,
|
||||
chunks.content,
|
||||
chunks.source,
|
||||
chunks.page,
|
||||
1 - (chunks.embedding <=> query_embedding) as similarity
|
||||
from chunks
|
||||
where filter_source is null or chunks.source = filter_source
|
||||
order by chunks.embedding <=> query_embedding
|
||||
limit match_count;
|
||||
$$;
|
||||
```
|
||||
|
||||
Voordeel: één call vanuit JS, optionele filters (bijv. per document).
|
||||
|
||||
---
|
||||
|
||||
## 6. Chunking strategieën
|
||||
|
||||
Hoe je documenten opknipt heeft enorme impact op RAG-kwaliteit.
|
||||
|
||||
### Drie aanpakken
|
||||
|
||||
| Strategie | Hoe | Wanneer |
|
||||
|-----------|-----|---------|
|
||||
| **Fixed size** | 500 tokens per chunk, 50 overlap | Default, simpelste |
|
||||
| **Recursive** | Splits op `\n\n` → `\n` → `.` → ` ` | Behoudt structuur |
|
||||
| **Semantic** | Embed zinnen, group similar | Beste kwaliteit |
|
||||
|
||||
### Wat is een goede chunk
|
||||
|
||||
- **~200-500 tokens** (~1000-2500 chars)
|
||||
- **Overlap 10-15%** — info aan grenzen niet verliezen
|
||||
- **Houdt semantisch geheel** — niet midden in zin knippen
|
||||
|
||||
Te kleine chunks → fragmentatie, AI mist context
|
||||
Te grote chunks → ruis verdunt relevant info
|
||||
|
||||
### Fixed size in JS
|
||||
|
||||
```typescript
|
||||
function chunkText(text: string, size = 500, overlap = 50): string[] {
|
||||
const chunks: string[] = [];
|
||||
let i = 0;
|
||||
while (i < text.length) {
|
||||
chunks.push(text.slice(i, i + size));
|
||||
i += size - overlap;
|
||||
}
|
||||
return chunks;
|
||||
}
|
||||
```
|
||||
|
||||
Voor productie: gebruik `langchain` of `llamaindex` recursive splitter — handelt edge cases beter.
|
||||
|
||||
### Contextual retrieval (Anthropic, 2024)
|
||||
|
||||
Nieuwe techniek: voor elke chunk genereert een LLM een kort 'context-stukje' dat de chunk in context van het document plaatst. Embed dat samen met de chunk. Resulteert in 30-50% betere retrieval. Voor productie de moeite, voor demo overkill.
|
||||
|
||||
---
|
||||
|
||||
## 7. Index pipeline in code
|
||||
|
||||
Volledig pattern voor PDF-upload → chunks → embeddings → DB:
|
||||
|
||||
```typescript
|
||||
// app/api/index/route.ts
|
||||
import { extractText } from "unpdf";
|
||||
import { embedMany } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
import { createClient } from "@supabase/supabase-js";
|
||||
|
||||
const supabase = createClient(
|
||||
process.env.NEXT_PUBLIC_SUPABASE_URL!,
|
||||
process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY!
|
||||
);
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const formData = await req.formData();
|
||||
const file = formData.get("file") as File;
|
||||
const buffer = new Uint8Array(await file.arrayBuffer());
|
||||
|
||||
// 1. Parse PDF
|
||||
const { text } = await extractText(buffer, { mergePages: true });
|
||||
|
||||
// 2. Chunk
|
||||
const chunks = chunkText(text, 500, 50);
|
||||
|
||||
// 3. Embed all chunks
|
||||
const { embeddings } = await embedMany({
|
||||
model: openai.textEmbeddingModel("text-embedding-3-small"),
|
||||
values: chunks,
|
||||
});
|
||||
|
||||
// 4. Insert in Supabase
|
||||
const { error } = await supabase.from("chunks").insert(
|
||||
chunks.map((content, i) => ({
|
||||
source: file.name,
|
||||
content,
|
||||
embedding: embeddings[i],
|
||||
}))
|
||||
);
|
||||
|
||||
if (error) return Response.json({ error: error.message }, { status: 500 });
|
||||
return Response.json({ chunks: chunks.length });
|
||||
}
|
||||
```
|
||||
|
||||
Belangrijk: `embedMany` doet alle chunks in één API-call. Veel sneller dan een loop.
|
||||
|
||||
### Kosten-indicatie
|
||||
|
||||
`text-embedding-3-small` = $0.02 per 1M tokens.
|
||||
|
||||
- 200-pagina PDF ≈ 80k tokens ≈ 160 chunks van 500 tokens
|
||||
- Embeddings: ~$0.0016 (minder dan een cent)
|
||||
|
||||
Indexing is goedkoop. Doe je één keer per document.
|
||||
|
||||
---
|
||||
|
||||
## 8. Query pipeline in code
|
||||
|
||||
```typescript
|
||||
// app/api/ask/route.ts
|
||||
import { embed, generateText } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const { question } = await req.json();
|
||||
|
||||
// 1. Embed de vraag
|
||||
const { embedding } = await embed({
|
||||
model: openai.textEmbeddingModel("text-embedding-3-small"),
|
||||
value: question,
|
||||
});
|
||||
|
||||
// 2. Similarity search
|
||||
const { data: chunks } = await supabase.rpc("match_chunks", {
|
||||
query_embedding: embedding,
|
||||
match_count: 5,
|
||||
});
|
||||
|
||||
if (!chunks?.length) {
|
||||
return Response.json({ answer: "Geen relevante info gevonden." });
|
||||
}
|
||||
|
||||
// 3. Build context
|
||||
const context = chunks
|
||||
.map((c, i) => `[Source ${i + 1}: ${c.source}, page ${c.page}]\n${c.content}`)
|
||||
.join("\n\n---\n\n");
|
||||
|
||||
// 4. Generate answer
|
||||
const { text } = await generateText({
|
||||
model: openai("gpt-4o-mini"),
|
||||
prompt: `Beantwoord de vraag op basis van deze context. Als geen antwoord in context staat, zeg dat.
|
||||
|
||||
CONTEXT:
|
||||
${context}
|
||||
|
||||
VRAAG: ${question}
|
||||
|
||||
ANTWOORD:`,
|
||||
});
|
||||
|
||||
return Response.json({ answer: text, sources: chunks });
|
||||
}
|
||||
```
|
||||
|
||||
Tips:
|
||||
- Geef bronnen mee in output — gebruiker kan verifiëren
|
||||
- `top-k = 5` is goede default, experimenteer per case
|
||||
- Stream de output (`streamText`) voor betere UX
|
||||
|
||||
---
|
||||
|
||||
## 9. Wanneer wel/niet RAG
|
||||
|
||||
### Wanneer WEL
|
||||
|
||||
- Veel documenten (>50 pagina's totaal)
|
||||
- Documenten veranderen vaak
|
||||
- Semantic search nodig (niet alleen exact match)
|
||||
- Privacy: data moet in jouw DB blijven
|
||||
- Bronvermelding is belangrijk
|
||||
|
||||
### Wanneer NIET
|
||||
|
||||
- Klein document (<10 pagina's) — gewoon in system prompt
|
||||
- Exacte data (prijzen, IDs) — gebruik tool-calls / SQL
|
||||
- Structured data (tabellen) — SQL is beter
|
||||
- 1 keer per dag bevraagd — overhead niet de moeite
|
||||
- Code-base — gebruik grep / tree-sitter, geen embeddings
|
||||
|
||||
### Hybrid is vaak best
|
||||
|
||||
In productie combineren teams:
|
||||
- **Semantic search** (RAG) voor concept-vragen
|
||||
- **Keyword/full-text** voor exacte termen
|
||||
- **SQL filter** voor metadata (datum, categorie)
|
||||
|
||||
```sql
|
||||
-- Voorbeeld hybrid in Supabase
|
||||
select content
|
||||
from chunks
|
||||
where source = 'product-handleiding.pdf' -- metadata filter
|
||||
and (
|
||||
content ilike '%firmware%' -- keyword
|
||||
or embedding <=> $1 < 0.5 -- semantic
|
||||
)
|
||||
order by embedding <=> $1
|
||||
limit 5;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bronnen
|
||||
|
||||
- **AI SDK Embeddings:** https://ai-sdk.dev/docs/ai-sdk-core/embeddings
|
||||
- **OpenAI Embeddings guide:** https://platform.openai.com/docs/guides/embeddings
|
||||
- **pgvector GitHub:** https://github.com/pgvector/pgvector
|
||||
- **Supabase pgvector docs:** https://supabase.com/docs/guides/database/extensions/pgvector
|
||||
- **Supabase vector search guide:** https://supabase.com/docs/guides/ai/vector-columns
|
||||
- **Anthropic Contextual Retrieval:** https://www.anthropic.com/news/contextual-retrieval
|
||||
- **unpdf (PDF parsing):** https://github.com/unjs/unpdf
|
||||
- **Cohere rerank:** https://docs.cohere.com/docs/reranking
|
||||
- **LlamaIndex RAG concepts:** https://docs.llamaindex.ai/en/stable/getting_started/concepts/
|
||||
- **Ragas (RAG evaluation):** https://docs.ragas.io/
|
||||
309
Les16-RAG-Embeddings/Les16-Lesstof.pdf
Normal file
309
Les16-RAG-Embeddings/Les16-Lesstof.pdf
Normal file
@@ -0,0 +1,309 @@
|
||||
%PDF-1.4
|
||||
%<25><><EFBFBD><EFBFBD> ReportLab Generated PDF document (opensource)
|
||||
1 0 obj
|
||||
<<
|
||||
/F1 2 0 R /F2 3 0 R /F3 7 0 R
|
||||
>>
|
||||
endobj
|
||||
2 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
3 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
4 0 obj
|
||||
<<
|
||||
/Contents 21 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
5 0 obj
|
||||
<<
|
||||
/Contents 22 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
6 0 obj
|
||||
<<
|
||||
/Contents 23 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
7 0 obj
|
||||
<<
|
||||
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
8 0 obj
|
||||
<<
|
||||
/Contents 24 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
9 0 obj
|
||||
<<
|
||||
/Contents 25 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
10 0 obj
|
||||
<<
|
||||
/Contents 26 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
11 0 obj
|
||||
<<
|
||||
/Contents 27 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
12 0 obj
|
||||
<<
|
||||
/Contents 28 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
13 0 obj
|
||||
<<
|
||||
/Contents 29 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
14 0 obj
|
||||
<<
|
||||
/Contents 30 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
15 0 obj
|
||||
<<
|
||||
/Contents 31 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
16 0 obj
|
||||
<<
|
||||
/Contents 32 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
17 0 obj
|
||||
<<
|
||||
/Contents 33 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
18 0 obj
|
||||
<<
|
||||
/PageMode /UseNone /Pages 20 0 R /Type /Catalog
|
||||
>>
|
||||
endobj
|
||||
19 0 obj
|
||||
<<
|
||||
/Author (NOVI Hogeschool Utrecht) /CreationDate (D:20260607091455+00'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260607091455+00'00') /Producer (ReportLab PDF Library - \(opensource\))
|
||||
/Subject (\(unspecified\)) /Title (Les 15 Lesstof) /Trapped /False
|
||||
>>
|
||||
endobj
|
||||
20 0 obj
|
||||
<<
|
||||
/Count 13 /Kids [ 4 0 R 5 0 R 6 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R
|
||||
15 0 R 16 0 R 17 0 R ] /Type /Pages
|
||||
>>
|
||||
endobj
|
||||
21 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1856
|
||||
>>
|
||||
stream
|
||||
Gatm<>?BQ=&:XAWR$XDmmYRs%/^&Rq6bC$N[bGT+]^TAM&Ia\:C^8\nqAm]0,fQ-eJ<NBY1c(a\o6k*c;#Fnu2-L$nR_t&m$lFcq"Yc:Q0`M,kk0ArR\E(Bi!>13HJ--+2q13#>Uu4fg"!0-me0iC__,ZFB'e&"MchEh<KnbNK%Kk]`l73[,@\-P>B>5RPF*-d^:Wd'lIhSG0i<;V-d:&F8*u`$3X9$jS'eZ3(o0&JEi994!mjITg>O!uTbd%BNXU6(S<cH'h&'$'(j0FWe8h]*UIB/P_?g4qD#4j@45<$:\p0F!u&&[//s$<!e4M\G3at@G8T>P61L;4":^>PEfL*U[=Q%BT4n-%Uek^bf.pT5FepZb\SOPh6(S4`Cti:NiY]H9e&S8Q-0+Op"Mg"XhE,((.kB8s0h?I_(%Ag1mWFlG7'\f'4+UV4"TTbp7NYKcq+c-0W,3gU=_1LWmqJ_2u[MQf[In2(W1SI78'Ho8jfI6n\A<0*RTc)T&l2e0(9K+<5e6[NUUGP8eV)e0rZ(fRL@jq@*e!,JD_1%SUN_Mbne6^hK#`8XM!$o/YMSG@aEn1d='rEu#/2KQVS^Onc/c0AOe(3-o:2_2^b"g_GW7dUF+i64T@k)W:\#qMQ5Qr\E6<\3`MQG\!(FU,Ba$<PpW%WoA%q*c($9`aPO_9tVV^7cT!S`,0\"?bNo\8eJG\ih?uPZed6j1BJ!Km!h*XT(1]##ig_1.U.;Z(u[;4hQ6S&bIcW!&Miha&Lf0e+U[9[HW_21$f)';kE4*>l;K9VL$mHEXs1O5S7`a."<UQq(R]*`n@M73/4aa!UFk/(I-)F?_9J#6,Df7=f%&2/6QGu4H*jOM_-Kgr7sgQYm@CN%*!`E&;!Xn)A53l:+#i4*=Q)gpuh<_..QA?Z&@_eQIh-o4B*u:OGUF`V-ebt::joCZ2+5+-grU!.p.?QK:8r?0&YLQ.8"iu_&MRZ\]1cQWpi-uaR)h.;%i&h>;?89RNsrq[#u^uTf*(9$eZ'<C-Q+Y=<teXO5[^a)<T?'JJgkchn1=r=nA&)qku@h)4nYZZrp>B;B'NWQZ2d@;WeACq[Oqccp1Gj4&cLsQ&GigcdQR!NM_\XAk'2X9dXjR$.`Y77clC:HaYtFCmJ6Z]>+"oBr-_VVNXK3N*EdJdq'nor9E<&3b,l7YOo,VYB&."]<D?rn%'665i$5<T9aeWZ5?).k$b'X_RZiAF\D,ADJec`<lV+n11%pk=N&`c^dY4WR_naOI]>/oLLo/Wfl&a*T#\B:N.EP[V(5NsQOmXerdZJ)C"3$L[4Zp^_:IarH&J#T'bEi7N9,_A0L`aYo+:"f_b+#"ro:K:g=g_oQg=8^iQfTG5lG'0[IL"JiJID03^@\m7X9XP=T@)@E+4dPbT\"h$o9BZ@cSt'%DbEGXeY`kIoPL/gk@tnN6pJe]TO%^\WAgHJ&nqNP2=cu;dJ',_XMYUcgX[W0@C[<,E:Aa2+G@[V=?IAF,Y&QnZh,hQX^0Yh1F%0Y`6;]*UDE,-!iCe8U?f,(\c?prt+<rQf=-u&i'q3O,NW(lG1KS0tO`;dMfWAPUa>d$d4G:ksqpbJJDh#1<Yl_]YEPn##kg.`[:a*\9\jn;oE-iA4S8$N^8g5JL%*XP5@QA?;jQoEas(DK>;K)^iB+$1F1g3,lAcU]I?DHIRlL7(i]KlU;P?0S:[\^[;2r\q%h.MY`Dt)K);ri"Q3_@_4Q6r:NE[sfrI;C>Eg_ZY3[k'TI(f8P8h6g%cn:Y%`8gn"@W.96edu1((ZaA:\<dV=9m5W3^mlZQ)(uql8g5E3!/:tGc!;.qn`)5A;`=R]m>uQkBgh_cB`dC(@#3:SGrTM3DCb~>endstream
|
||||
endobj
|
||||
22 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 401
|
||||
>>
|
||||
stream
|
||||
Gas2Eb>,r/&4Q?hMRtF)R!KaeZ>9[ZPq&G)@)\uMmK9j',SAppo4XCe6#8OhB?m$hAVdS#4gQ@e9/2hj=<cRO'e,qOgo&P$7++W,1amMk:*_6ERg8%5b+T946m`<i/30c(*0YH#K+)nRZ:4#_)+P+@7TcQRNjWlhW]haSASA2:)jC3&n,6LdQ]UJ!JnDIo/:>;4:hOOE8?fN@kn0]2Vq2(nMQO(f>u;U2;O9T%eTd;+@_0(?<n4parB'HF[m(S!8"\Bl83`2rRpH32;Nc.K]AtVn"kEs_W\-1Mp3F@-kA/->B,1F1Q)KZG3A"4O,$^'"\soZn7C9&8F!df2LUaD[-_u=&J8.b*F*U*\0ooGbL#>d)M13Ba-Zo4r1gctiAWc.>^V[2.L$/c*g5gp~>endstream
|
||||
endobj
|
||||
23 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1247
|
||||
>>
|
||||
stream
|
||||
GauHJ9lo&I&A@[:lm4Hf7mR-:!5[OIBi8^nP;XF.^%_Y+O@5!.'b[f-q=S$D+/1>;an/D&(9o"UiF28"!<s6&/Gp91JDYBlo8Wk3OKX3],_>kV]Zm3)&`JnNM#\/=)J:$\(Ag=02N\.dM9L4a5gbgp1kA%LU/\"?UL0Iq'H.)>Rtq):,u%A@6nojM]^Uu=j;b'mLlaB=mbdV7SmFlscOOZ4?Ne7YXC'J5)CL1nOUB2(0`1A&k69Yk)q,SW\ustTCJ;!UE6ie:[u8kID-OHoXPe<oBWCp52qIp/q#f\:I]a!+P&MV,hGa@HV?J:H+0DTq*UI-Z;d'Y#"Qq#LF9%K:-^mq%ig?UQ(Z,[H'OtNMSB++852q09c*=I0GY0AYe>//+IT/X9.f$GA$tG0.m)'$+N-VA,/&Q@pJ.XkTh3a>?cqtq%Tj:(qhm2iH3Wd]KQ6N&9()7`4akh;J`an1"[#K[7r28<G2t+D&Jp8S=]f[WG`[Kf'nJE,&`_*26ASeY*KQU^Prg_oiJ=dF&M=21q1=Y\)r&`rsd+$7]Qp%OXK>2@)JGN*mXo"5)A1C)<I$r95rrJXD'FdW,4YL?rI@C\iDAR7I3;K=(eK"9d2kBnUnMZ.9&h,C8#%:fRJ+c\\:OV,N3J1=;*.9"J$a%n3&Qk^R=9CWpD:pPLh9BA0C$UZ1\jdJqF?V6nL5TCZ$\p.t49T&mm:odn>qd>Id!>f;dTU.)-bUm'_X__TKXqqdjG.`s0a_<2Qt:F_MmR8._LpG5TQdmq=HtkC`+Y]8KD)KM3K8/NZ7..>?4XUbGVD=IeA36-,jejNPOk+k4RZF^f%E2Ze#O`KqZOfA)s=dR1B(1mmS%1])8P*ref>]>09d+7SW=WUCL&kAi1N%#/s'?DV,p5t,lYhLr22M^8prMG)Ck-<i>ZP,-\?:s0!!FeVI;G@jWd+e"OIri5@.8*Sr",7]il=:=28OO/9jDq%Z78Kq"#35p5i\L#_t*f$4T*N7I6#ui>g%I.A/X>8a;j9/&QEi3J.Td7`T\'^A+XQ6u:;fKmKNr*s,fuIR*X#"4%%WnBVQrk`rERMm\Y*Ar>U,SoMCrAGGKo&ZuR"n/+.!7\dRAr,h)[::$9tJG5&$3aeKgY)*d'^7sXg?nGes<U!_dP4C/djLM#_`_Ws_Vf`.nF&A5BbWdYCW#/4@h1KDp7!C1&4IBi^Xq,U+(pa^K$k""\Da7AUp_%GX\p:N^4/!]sm]ttu.YR"m6,-l$~>endstream
|
||||
endobj
|
||||
24 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1794
|
||||
>>
|
||||
stream
|
||||
Gau`SgN)%,&:N_CltH6gA1Rud!>%r&Q_\e&[n5:1Wq-ebJX0]32D5D/4&%Nq[s&.?&>dI644@nd,Iq'L^$tAt`IQs;g#%&bn72aPV[ODtff:`P"O*ITd4BMa&?^7',`kjO0]kX[Y^JgF$jr)RW0kU61l[<O":38.f!,F>l_92(:`26B(bd(pU6-3V1m:P40@knHEK6n"O\3UIlT>,glX!7ak8h.u4pZl]X:*1GL4W5K,71[r`\2g%3JJX!V5`VoAXUC'9kEH'%@hG[_;jRtj18"O`[p&pgmG0Jjec]4_!D\G%pN6?Hm(J3kh[ZcF&N1)J;:]bcQ2AUaYGF\&-JG(R+!*In]oO<C(c7X_Imk$_LiC5UI1`.JiqtS\5P'(!ZhqGKc\=Fm^*+*OlA`bluhHAQf/8"N3uJg'Vetc_./8El$*GZ_&OK^ca//\?+.I=4V79ejoJb7Ol;"<Z<Q\5(dpOu/cs4qp?`q"oP)Jh93t&/7utJ2A!CrWGR+^t6S5]oA:3lO#]j_:4q@(I`K(?*'#.u\MikIfphu<"dMe&n^uT-?Yt0f7GPenC@d:MnIV8hIe"SU%S$lL7lO^Yf51L:c7mg4gD/7Ng?l+c?be,C*:>3UE*Fk;58j:*d;SZX5L'ih(4?$7Lc#m;:e:[l%1'07Q<B"<cZoA&@#E>^cQXh>Rh#VOtgm7;c@bS%JpnK(I,s/OK_Hg@:5X^b#/7+VG6IU,:>3+G!,#lp&.+,`R4Rb+_e74GX$e<EWTc@FdK?Lc'nfE#^6EMnE^.2?;DUUH[?/'*CYLbMU[BJE_FYrF5SWjX.[Z%#o(a"s/1QHSkOsoDH&TVu,i!N'OBA,V]Jq\b<Bh$I_-Ju[EN0td'KG#F=!.jeD&457@DM5\4Nk)JleCnRr\14a&B'Ia4\M+ILaXXtl;k4BVCbTWlhT&*8ZmJO?e!q^%U]nCf$WCV.Z.K<a_5"4<kKI*:q%kr4F-?sWc$ruG-`1(9/A8\W95rKD1QAq#"kMTj=<YRDY$MCH9JJ7Db-G2Eb*t;FQ)hagHM'h`-$t\^PE?adFE`r'MKR9E/\f:qI:J]+jhbPr.;lQ^[pTdW'7/gq5p9:pHC]$6B8;N>X\-NUWK=W\.57?qA%,"m]sq>&WR>\c2XbJ&#,#pgZ.s`SVIhcPR:7G=;YI,GBtdf6,7=[A]N>$)XD\tFGU*Tm3UAW\,Xl-82,L/NZO_V0/RrDrdM6_2rb*X-':FN;3`!&0+a%ST4_%8]S`Wuf>N:s]fFq>aI;$oilg0o_ZqYGIk?3CZ\&\)X/ag,rO6<&\HHsD+Sp/maY]u:(aaAPC-1d2,71`\bU<M)sMdXs;Zc@3t-rPAPj%f&GPr)bW\_V&!qRjI5P&._0d[Nt'gC!re^R*SF[,rNt1<6@X[.CuNieH?^"[p4_^0'uL=lE?bYAL&efrB`#pKH*eJKm=j55]\ULZC0"Q3q8cs*Ykq@>m9XXG5a@'0&f<Wr4oo7-KlJ#@>i=,Q>Q@,@3^g@!$8O59<<3_`n_FFYlb_<EV3Kac3A,mdMDL5/LqlYK@hS8NQ_Em^[jF8XPZM'<k`0,"XrApIREF(KXN_h?OpR(h]p%jR$.GKL.-!.\E%2Nl-$E`9lEL+`.D[4k&g,VrQ1mmQ*CWYfA1-'-XU@mWM2lV"KRIVl!BPg=W0E/.duC0QPY$Vnl_qcPT.B+i-1db"hK(N=Mlb,eGsq!!C&e!$h9OfCRr(HI!HA4[1Zgb/Urr#u8N`d7t\3l%AkoM1^'<m=8B+D$O'*K"%LcF9&)BdC\e]?3Lo<%1!g:Nr~>endstream
|
||||
endobj
|
||||
25 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1512
|
||||
>>
|
||||
stream
|
||||
GauHK>>s9G'RnZ;3"LFV1A,8J>"t4+O8o_1-jACJg>PJ_(F.9QO^q#[l[*^i[Ub_"#AbW!IEo=b4*tjW5k2gns$7(_IK=AI)))PY)dnh"=Id7U_YNI.Y.6%t&7MO,3"SlMIupe&"I(7eiuS:i?".%FdP,-hB7QbKlS983$TG70#/o'A2%k^R8fPI?08Sa1\!BErm;;,Ip6j>kO6FSE1p_WU(eV#e</2qKThN!((,=$^3WF1&+4HM"KA/5Bi3so:_/05)S@``Z1-`kO0JDc*%den.dP%k"MH@$lK>m.sY5uA%S1$5_.K6<kJLSB-?DKM;m'$C<9RKY=S."t'$`&&XW(RR3K"?1B4!!_pFGpf09+A/D'rc,`[E.;ZTi@Js)E3Y?,g=ed\>!/p_!0G>L)J!'W,B^E3DVqr/QaUR):0u=BOH0/3=NZ6rKjknU>7GQ6[L=-I-?N*PWKcck(.U><&%CelD&iM>fKMR]UG3749e<q.,fo#!=Wt%;5QB00fTudC,^';#PbmIpB]Zr0-c'o(q3<15M;]#1W[HnN;P4XrgJXq/mpQs>o`3&4UReKdr!BAQYDF\8eIDNM%Hl75bto0D3UjLMp(3WOk*sJEnH2[1a[V3Tefn`@5+9b$sd/LZB>+AT?'N?4Al>P-^Z11AclIF8[ap>n>6XnDM@U(c,7J[+5N<DNS;G&hF"QnF.9s4Fo"+'cfPXnZiG:JC(Gk"jj;lZl_o>h'VhS1h^3SX;M\Kt_7sOFEZIhEG%NW>=6g\Er1m>uaWA&t[DYOsc&g>q->=7m;4+,RQW(P=l"JR.=pIp7[/S!%STV@4_+e82aV)GnZ?7!#G)pXNet4j(Q;/NA=d`qfj6$o9g__XtUH#\+;)KYP3<kg73CL\nRko%+h=q]#'kZdNGi_sq%Pr9@&S[Lb`Cc=!263`ZgG&qo^7R!LSBf)uiK2%fbh*?uNC<.+Oe:E5CF1U90rmTKG0O99dR^J>MIEp]p=]_P@<u$,Z1U1__2Jt0@J<H-Fa(7Nr1JS*NP`;S$_M9Emqi^"PD3J2lBc#8me'8^PaDf)L(ncY8$P=?HN<5ollJ<OfuPhh9\dHiR$Zjime(DaC7DgQVuHFcs)4<HqJsg7]q3@D/[X<BLrS"T4cCKiD1R[)Z<4Vp?#3l0m!h.ZM0r'^?0=4NJu,@E>7!L0T"(d>pE]]TA"bcB9%R!'UH&9^8*R1agT.:oV^94Gq(Qufldi27>DjJ8BW9W;Fo=fD.]GL70s+.-oNfTiIm$Y/6+(`X'nRrL_2W,]3X`M6K^'Z%_;-t\n29ULBYRV/]:5Aom;6_4HT(iikj0pjJW,VKmN1mXj5k)FPSDcWbAKP:_qTB"6C:,:2pGo9%Ln!.g#P?&\:Psi]=*u"bpP*M<(%,'T%U<-:6.Ri&2qA0W;%F1k(ma3cZZi-P?#$XG<!]qTC8<\>C7TM.la^k35.L>KA^pDgul;W3;U;c&R9%@?QL7o_YZQ*durN(mBueAH?+27!;46n^]rsD"^7//~>endstream
|
||||
endobj
|
||||
26 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1839
|
||||
>>
|
||||
stream
|
||||
Gb!#[D/\/e&H8h>E@Y5.l"66W8;YL9!H-A0"gY!YDS$D`<^gV:-$jPV2/Q9n^O8Vc:i`)Y)%(Jl%8/hR*ZiGd3M%'ar3H/`8c]u-iPAFsN5P8jQllI-L[\j>S3)CNg_UTZX/HtV+G";l:b8+FUdr(J#7M=>M=LfU`J@EC!fh`Dc[7,tSD8<*&Ds7s&jR.pT(4f;?/fja6OP)$0=^<)`huLT!V+n-0[/[7@+^+>PHq-i+I#H)_DHtoMkbLBiV._nb@m=,$,!AI*A&e/EULa*q05f'EEVNd*9[d$jJu6meK.C[@=$gi28C#['8b\MiudP!Qit'd<IWB3^'kft(4nK\H%g%f!I;<ee>J260(j3;[P#)Mikd*FGf#*_iK?%,es8;N-^4rqY]84e&qp'%+'N4,-ZZ9tDjY"X5Zq$9r;m9@gLX)$)aJ%l0Ym*?E>UlNrBk=NW#[1)BHKZp%3&jDCo9"TH/pNIRT&A'D#K!aL-%8<4ec,+%(;*l_A2)`0Z8+!<:uQ6_7gK3/G%EX!8jjI"2["e%3nu*]NrF)4a@74GbpT`e%7$d"k6H.HjG^I@_c(ta]bLi`.:cB^L@5=kpNV#8EE%bK@M^.I7J98f:Sao;r'Y%:e!,XBq;)/&hdjDTX,*Zo8gKXEg,f](N<6)+o7A0$!ohb]^)K*IDN,2I^$T[mZ*mBcY2XS;W6&mlseJ`...bG,C=9,<iZW"(ZEV&VN10E-C%_<iWqq=a51@uSO]WOrm7CN5*==gnqk[T.;Gg,(6F(DFma(]IF`SI&QfE-oaN-"?`CkOg4Re%/>FrMc[kep[f65AqV,dCk!Dm_!PM6;An8n9bt-j)INsB0#>a:Zbk%dG5*L.RY"O//o>V)Er#97nU=AZg7*CYE))CTKU_Kj,9P5Xgf%'KIhKE"]95KWN*qi0ON#>qLB=^C]mb_<PqjT&goR>is=/A,_(nijLo/Kr>SAAr[Td/p29C5Vt9?HXf_Q.g5DO#g8M?p'X#?($nF5TtWP_-HW/sfTGb&'5n#cu!9_)AA/XG-@UZleT=90rDB$J/iHJ_l\uZJ83dTmd39XZ]MeQT@-pCIn'-:*<5&W,qi3];2[1YfD$GLBWo^)CJr*='qWM@1b*9ZRT=AcPi8@;!t4!SnWW$]%6C?GucFA$o9k*r8V-6%QtrYW_[nHbU^b9E#`>T"r@Qr!oG-.^'G.WV>RlFmoX'k#$#o<BWNe\$DJePR8Oa4;WO7tn;K\c%HNLj9k.;W\mF8^8e$U&N0A,;ML2'&n[on`mj%JD9@>$D/aZfA6&P/``&>#$h4aYBFds$piVe2GS9m;dDp3n<1f+nP@U)N^r$=Y4CP+bUIuKAa^o$e1FmSPAGfK,tJ?>W#Z#1=+%El@5"]H+NWg'!1=`]7'``RcGZ9heJCWrpkl4$+7b?.gb,taKdEOAZ7TLb@e(ZsWfJOu#4Plonhc8O'@Sk=2@[3Z642pjAhPN+>J:]M6'/l$JYl)\CI#S45%k-[HeS(5fbRo6ETM#mXj0@`:)[BqFSFB(W'do@#]\&eH:EtTT;rqj@Sn1B5"_1m6-iJW8Kd^=@miB'E_a[TYi-L]]Dc26'YHSF07on2j"UQX-'QUt\SVFnI1&)kDNh;pF[d3c!!jgDY5e6R)>449s`_$6nWBS+?LDEVE:(iP0j7b1YU^nkAQ4KLg6,)K+AG/3/;\E3cIn2L*YRT%]=2aKPiH80j=kB)28nCk^'pM2!BP(p$jBJtb;7HEEG&<V8c=bZ;[Vg4fW7X&IN%"I=-\@K.+g\^m!BB_"/`cnU6P@sc3pFEuPPZ=5d4%:'YJ%%Mj-827bO,dNgm!neQ9t0'Y"-=K)"T~>endstream
|
||||
endobj
|
||||
27 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1881
|
||||
>>
|
||||
stream
|
||||
Gb!#[gMYb*&:N/3bX2at7E^i.DqZ/W]bE'FmC$5;qdL[EAD1'/99(9qakuQKOsi=LV_<sN%#Z*fLVr*0S-]:TJ'@q)>C^)DnFAF=`aQR_-A[3J48<;,>ss>rY(Zuk)]C(D8.rNFeEa%9WJF/k_SAuBN#!NE,K:u536p!ABrEVbSja'D<`C"=`%\0L\hu]!MK<$+&?/WLF+3SMEUG,@p"lC3h&!nH"R@u]:h)d>P?PhN$a]SWI=;QBWjQ]L9iVqt:KuRie&$]X%BnX&Gh"Wre(Q,GDD,j(mC'H>:LO/:k0Mgd'7:r_Om(XFr]Aj7PCS^kX]?J/LNK)R.Hh)`7uZ[4Uj2@,VM<#DeT_i?#!@bt0I@Q-.+h!oP-,[=7Dq'W<1&N%RspSE9Ii<7aC.aX"WK@e@E_"@;hF$B#m!,"h3d-9@*$^2Tj:%phm2bu%6ubOP!sKbU;0NCZC`,F1"ggfD'I:TpX^GabL_b9T"KrJSfX(?A!CrMGRP"#@j?st1-%?2_R_GYHqs0ra%luUNUI_#''/E\rbd-uZaQkW0B(B/Q_(%54[oU?cp3;Ckn4gP#KltpoTth(J'a\Q>H_K/Tk`;(NC:-TXY$$T.LEa.fDYddZ"2^Dg"7bt/+W$Hd]C\HD/Dpdi8^A79"_?tETIMM-uL&DG^+DMon:;K5^4*6D'ZLVZ9ti2OYG9Z*).T1-pe*f;i;O5A5TQc/Vm8]H)P+<VJ7-h!7%t9"pGNh"eiqrj/b.NnsYSHFBm#:er4sKUTaXoEAqD!:>3^LGZN5(=IYDj#FS/DYhG`L/?<XAg@9h>?U7i&BYa8j5f^AVn'5/=:uHYS,nBEq;0Y_IXU^Y6e$HnX[b7#1%4l*]Q=2Cfe1WT[aHmH%BDK-DFe0AQX7L13?c0FfIXVE[9#?f,4^:qQlcaNk/V_/5U"gI'&XW^#.ha_Sm*`9^]ZSFkaCM)K/-Y0iDK8dCGRWt`qk+QGKJC,pZ.h(H>it,d@CcPGba=@>ioCB*eAQXM#']tg/Pf=;W6Mf[C")GZQ?JP]$g69R`GBPb0VXXs`Oa6"Mj#AJMIWAjcHX6hicm&GRSS3nQV2Z/aPMqL7o$.8G+78G/l8%Cq1]"l,M0p%1op,R4frnPU%d"m(((0Vl]B+=U8MW>jgq?L('gs7N&t+<i2eNm%b9YN$3bUW%$@g\9P^5%NElkD,agAG.J,<3Gcq1);Bn621RsAaU.<-K*MuYgPi5!hCYB<_I]<&tK+33R#%dGVQHNKZ_<d\*/k(A7R,P=:lT"/m(qb3HSZ_=R5%$a\)C1+.:`Tag(!_,0Z8uS-RY;[&]Op!#[4lt!rmM[5Ci2CU;rQM?NfM??jL.hLCm/M!8D^82q:b<3dl&cbpeoY]Z6A^P\#ZP[9JT/Nd\\FRGSP&/hOIS'JN7%=7d1XEV(E$Y__NeuQ_G<,jI%mVXaqqe/$/':97h%4MX0F4ebuC@7i=IF])(!nq7_Nc(,iUTjuhPgmS_PqpF3/9ab5ff=!]dtK1Q[9`FS[Tg7mK*I>Q+,=)B\hqnm=Wp\u9V3-F=cnim\0UPSq'?n1RK<QX`@'R-XGr1HpB#E'DmTehT'@%ee^*S:.k`E`clp1\\<;UoqlUaU0?oX=0A:V1^o<-Y;E"2Mn4f%6fY;ro5MAm-]-<prMc8(Bu;%,rIlTM0-X@iBGc2q5:lf4EF/ZD0KGX:_\t^e2MdHB;khe'Eg/U$q's]L_A$]nTNaV>'oBGA$knCb#b`6BRT"]dl%1MZT>b6YQWuSpXVEd+H4uF3auP0:rd"IYXWSmIC''#f'Lp0ZlPsaI/ZLroGJf(+7E)0mJa4/-&^n[=+;5:J2+d=4Ksfd:_)#(^D-P4t'&($J\-5V>$,+B3LN^H#d#%5kH_8FrbN~>endstream
|
||||
endobj
|
||||
28 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1594
|
||||
>>
|
||||
stream
|
||||
GauHKD0+/q&H8h>Z!Z4)W,pabmS0O""pue59I[ns5@?Y\j^O.hP*+gYT``'CH`,I@Fu8T.L_EEBm6:F6hX96T!Q>T@hoQ)\nH/-N0OPk9kJ.34Y]pIM(W".BE5f5H5XIiUL]`K#pB0M,a?_;M+s&Hm<"l#1#fDHQOTCJg=u-VaL%BWl"/6O\7K>/:.\gnGLI?-BV?N)S75smDI`BcnLhbTdc/ohY:kH)Mor+Q,PY!H8Z%:SHYPrnZc\e!iP''W4n4-d<A6m*9Et+2%+$Zd*[&WL24L(52/%-Cdf13(?!/l3Led,Ug:*e9"NrRO.^a.j7Wf-HQp@ifsjtW58c>/AQ#GcT;<)Z0$OD<R/3s-OW(`3nn,nt.s/kL-=Y@RadKMl5JCYZ9pJhX>%GXZd28^O["%5Wr'W.$0^\O]&J*O8/3-dXFj8?f_P!etXHn<3K1['u2$FC:6i6Vk!fmbJ_*ScnApkEPj#>CZ[kjc3,C7800lUFfu?E5Qo+$unWZNScKKg)%Vp0]0O'n<M%T#euAf14hr^\(C,!=mtp>1g`OKQZ6kDWVt]A2VE`\5X+gsWu1<s@A3lR6u"-^@kRV8CSToX1uBZI,$0WblA]S[,g7Md(O(aYJYPDJHbuYan9q*-])_6%3(>4QN;)>XM?AH8@Bs<RFb5+^H#@IZ&uK!@,KCofbY;i5EKYdfY#(seVoZ::P#A6:$_(4=O;*RF>A!DKJo@^e`1G#]XIc2tc.6LZO7qmJ4bR/#ND4S-Q!R4*CW0)Y=^"m'Cge9L0c^3%T@;S$Y)8r9S)Dq.4LXQbG?2>OhBhi4=!"(Q2(%];cKd<T+P6rah7Ud1,H'>ZF'+^HDX0WOB(5mP4h7NQL*;2-Wu0["l*u5T.XQ_r+0a1Qi?=L#a7C_O-^2@JK/^OL7)Dr*0Uj5nnL9)h/CiALgUB"K>cr9oP"ln\b:t%q0&O)k@r"UcYVq8:@F$JMY)-D!(jfJAjC8ZYZi!9!%f-Ar1YH=Lcu(Q9W%_F2XUgT5'0nGneCqJCfY@p"fOUK/EWreQSC6GkXLt+?Wu3I$?Pu9EmUVK'>ZT%CK4JpZU#4/UcITc&pf5d"nC97Whq`J.%2W`-?9fK?Ss2/5N;<&$(T,IGEr?r'rgt0K]*6JnYd/%[Zi.W$WNBTmStWV1#N*HPnoVM"j8?`n'p8@MWcg?H*1[nD#K,"p&#JdeN$%j4`SghZ;1EGk`K,Ck`"Lo8(Rm['h6`u,/IR6*:2sD$qj&F*:d&,>SP0[gCCKAe/d1cBrE;UBPTb/,%/89[oaL[3KGkoee>J9D]qtlK*Gi:&cO<"aJSV;NGVe.C*W3eJ#cIR^^5,2VFE,>0;poU@m!P-IF^!j#CL,Rgol=*npQ+`5_f9PMCudrG^$30k^]!3`#l`\1%X!N#&BRD=0a*RPGuZeHWXf$q42cIBm_R^E\.b5Rm?%d<@D8Y<7jle#L0EO2euE#Xe')9N9]7@</s=H@X*QqrLXa7FB.XV#5*507=uXhQ<L+sIU$"u5#9U85dILt&T9)Hpg@VVJqQL;#E>RjGKJ4Z4jLg-?EjmmGKQdAe[aM"SG@eDVeV'Xal867\.qf4f!PeAK?2~>endstream
|
||||
endobj
|
||||
29 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1697
|
||||
>>
|
||||
stream
|
||||
Gasap>Ar7S'RnZ;3!`R#m8A\7GZ0>,B[I#9eue?%D@c=['Ua-okS?bX"\&:BG:;b=['^hX-?XG*SaVX^a^YMh5HO`WEe#N134iS63%R#(";.kc$."pR9gH5J>"Bd0Tjblj7%sL1H5dP@'i>R((`990d/nu0`0eUGa)P9=%CLo7K7F!MMJ=Ad(abMF?]>SQiZ(;n,*e&RNplc]7kZ'dB88>B5D#R6l]W5a6U<,a>/E%M`?2;_IK@d<X_j6%ZHV6D>MJ5=Kb`bh*kusp\7sBg[;1#%hqP66Ea,pF@JLD!%pZ1R^(?f*UplEbdPeld@+2LE`l#NN[qQFA1rFTkF#j:Mh@nGhO>*IKfAb_A#9cj$$&'!!npjBm];$qc\0dFI1UD)MHO7QricC"tI$.mE@>>`?*7SYZcnCP9@Y`2)N!'>PJcpjX(,1-ZVO)B-U19E5TG*4l+%DFli%^3ok"0M,FII4l^[u.T:-5"k'f!BaAkU,liJ-k6(ug9>RVp7>Cg@><dsSWqiN!GQ,5c5(V[+'3DYNX-'PmRXWjK'7>$+<'8BS>'-3f%M'Ib7g0eeKh\9l6HMkUjeDE#F\fK9^Kqjf\"Z9T8l##;%8HR_>ceT6WdI$.mk`#?ZPaucQL`_"7>gZfh"`Tj!`#,;?qV?a"(D9V?(S\XG'b=TQE3REUu5CUael@)YE+[k.Dd;LH8kEn&c!Y^=8OSM%eFp;qdS]+TTQFaIcX34da8_\Xi+;ch"79o.$d_%'q(-[BLf[b&j0!U+7f[OIB-bH*VDbbaAK#TMs:?<M+r6h`tO,HdY=I^F$@Q;%b[0'0g&j[biKsMl5E,kKK878bSOu=m"Gl?p2j_@ZIRreARhX#(-Y3s3j_m,Ua7&!^D=?tg]9adGrcC`*n%$)P1&>q+OmPeZ1Xqn)87hW\FJ]PogIF'gW#WTV]X_h-t'5'HoRW/EGfKBG?7(<'eJ1l,9NNEf(YTMe?>.!F3?,rR<$d[Wt+L/78ko:`Diu(CJ.b=J7I"p4QA=/[NG#euHitT1I>Al+M.CM6l[bb/s4W[Clq@/>mCOkAh*,2h,._47Jf*ucc3hD8iGm]^4pN8SX_](tlkYAFSPh0WSD<<8XqHba-;*623;g9Bs4-sa/j9MueOt\3$iG>8#O,Z_"5(9i=ilej:`,.-VO,o,),&,HW)KS+_)UHH#$G8&T..j<,*`jbK"m4ZN6#d5\C@]cUJ6<r5;\9VY(d?p[2!(l8m(!p=&a/j%#2$gq;A2Q[c*52F31heC^Zg5<OG"l/e7JipFcnsO,RfM>3?Q6\>sjVgNN<#(s4Xon]"iBbWDGdE]PG[,8]4EnTFI(;@bY5fJ4,.CU#1=^.\6u<'olAc9lTRL@Q=[1I#/(<(6>soZF"X`/<7mCr_rgqUQFF:QDlW]&+c5L<7<(dc7WYi#)uD<B=JmN&>^^qlmFb.S#f,u/ff[CLQl0knMT\Sr&="f-?'.g4FeWcB8cFa\R:+'BeDi-Fh6P*m-B&Q0c8P+`!9Y8dr*$Wqpq$8h1a%m?OpZF-(MB4`S$\Vn=!nDc],*I@Lh>n[2:HD/$VVu\qP2;]:'6\1B3CFDE0'o$bY::<q-r35J=k-k)5qbd$E:lE9GpkVWaZpGW+mi\AegDjX5r)/s^mc7IEd!)V-X/Dn+Eg7"DGEF2$m@)(`4m?!iq0"[>@QfSo02bGjf%pOV7P(^W+:~>endstream
|
||||
endobj
|
||||
30 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1584
|
||||
>>
|
||||
stream
|
||||
GauHKD/\/e&H8h>E<u9Sl!g*b,3>L#>&<R5'\fq)`IN5I'U`sj=>,ftUd^CF^6>KS2bmWE6S63M:?6KB*M4Qj"r&`Ar'h".b6?*r0jkt:(Vh;YY]pPj0"bt\E1^pFJ-\M2^juW_r]ma^;G]RI"!0-M<"l#Q8Ag6<JH:ekZt:3`$Q%,O#"8>f)[XG:jW"69GX6`&dBt7BpOfZi\,8oo0-G_e%V`L-Jmf2@#1]IU8LRM@_'4HPmsf_0"2AD_>=Or^[Xn0Q@AD!QQCM`8nRhWl/E=%A/,7*?fM.YZ2Q`!JnHfuWC0r35.1F$poIg,I>67`R\$F./*:0<V3_i%MF9H]C8aVUehD3X2&0\!la`oT$T"rPe+arTBic.ClZaXd6XpLrQN,7:V]d.%?k@9G8qF"Y._ZhoUePNq.^j^id*a:>MV^Kh/+sbgK2]flGCj8Nu+PM1!4?B_!i'U;G*ScE)MELG^]gbuojlV$cWYSC*?D!Tc+/:i+P]4OM!/A##.+?D.)9iOt0$Rb\8,.Oc*t\D_?ZMlCJAU-Y5Mb&Q$fl;F2fWRU:*Ib\p5g,e&k$M/;_82C&JiQf^H5:L@_d:7gpp-AWO47ihR;sMU%Q4;eCUToW2+`U0jUEm)ld$:KEp^Y1U\/0Q.EA2^q$2B;pV,G+OomWlTlIGH;+Vu3r#^$@>5W_^pP1I6o-[o8jYeibS]06Lk#VuE>06[&5HLCcXmaNq(0bM=]f"TMJQL";O..2JKBA^,+&IgZF$^k%^*:LXPtdbH[$%ZD&??aP$O8IAX4qrbN#3)+AtfW%:fqDGV<Y_H)QH<a#5reX&Y[m8"L[SEcMBkWc)q[2rS@ak9Gul$"6O@'pq@AI+_&8LX7NP_Dt#Ng[qKcE@SGCGTN0:'R@'U*$o#.?%-n/N_\O#!$0$U$G)oeJNAnPbJug;q2g7CBS!L,>0T@(*5Igf'uO8VhiJRX`<,EKapQ$t;q#_en@,]q.nk\UQm?+#ZC"$-&20+'q9K<D;FQtV]("&FH'`B+aMJI3nW3$cNq\?oNTpH"jYEq[)in]UJ+b^>+L%NmGt+3b:K`X&4[K_X:8_%B3'*EE>V(Pk\9^?,)*]6A)T-dkdNiGOakEW##V3MP1J7FV[n;k<.r,gIaF+P$DUU;Z0\1M,Q.\!NdhkG3TEV<N0=5Genfg_I<7';k)lD/&Q5(pg"]TGG@QgRDPg3RKh8?p!f7tOYfCc4e"a>H>^ph)g5]CVT;26-Xk5)U[FLJf.NEOnK>GUbT/TXUtD:bUG#h/@q(@u,N:;CSB*3+4Lao9OXW.+?<:QOGJ]-fGjk%bm[Od!@$B>$Q+(3*[+G<Ch4%-K\,BEfQcH/5Uu4X0?UYhaO+_K<.+4_!ega`gl?RF[F^`5@UEGdQ^`4Toj\"4^83@.N%*$TKfF1pnbX[Q-2&/sL;/PNpkn(Ye`-(3"u-]<OeP&N=5I9;.#S'fLK)+)\jcq^%s-olq,E^*HZ\5/"fZ5W'U;9MGMfD4T@FKpB=W[i6$\rL@@1lI3>`UgP,cgfC#(s#Yu8^T-34r\FkEIb#lXo@tmIa)HZ(&"MCsiZFriIm9TFg0ES7#OX682Z~>endstream
|
||||
endobj
|
||||
31 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1632
|
||||
>>
|
||||
stream
|
||||
GauHKD/\/e&H8h>EQBN*Mcl4;7*7kUADOg#kgjnL2II(m-o,;5[7"#XPJ/1>I4%X]PfB],I;!.Znqhbhgtd$ARK:Bcg&GarnA?.c5@_JgXKR-\,%1?hITh*e4MjH<70(8-e;lSNaD_*Og3No*+UKA70i1g^,m=RnLf0?:Nq9&W&h4>lO$!V7L/]W'7'Rk6iP#%N#s$&&"UW%"qL!b<:UTg=')([RB(nSF*5Qs!*3(XtAUt7tS\sHr(Oo`+04q9NFD&f=rbO'TmM^mAQZ.DY4Kn(PDY+-^eCT"bb$5b-.BMFBqGFlU"CLq]TJ,jgrc;?I$NP*fA$.lFhihnX,f&%8kTpFU5RMe_<)Q/4T1Ves(F/AD(`DFn:kXpW4<WITZgBn.$)Ks_Cg?Y."CX(H;bJg'8Ng_(#Gr>RTq!RKEDZ]GgLo=Z7pn$H88rh1HOSf;nKi<Ce?+m[FC_Ys090R-fsrj-4S<lh'ifU'\UXOY'ou@585udR8^8/!>VYYV/hkX%:c;t6,VOgn@K-0g_%/,H7/21X_+DEd'^d">5L+ab2QbW;!jjR&g"mIo;([G5F]rJE&%^(mNFSX8'l;:GY%V$;Ml.o:1gm`T6bGBs(TcXqp]c%m`\"1bFENAX;'&&39rc!$P#6<2MqHiIBj+);eJ$e"T6ioIXoUN:h@kdT'1\8P&g<s#b,If=@o@73-!4f/j@]mBCfoatZnjd_N^[Xe3<mc\:udmCPu=U'CH@'pENN9/C@m(;>l4>l2ripq@JUPg"OU_jN(Cl)8!u/.I+EG'e.d$-<O.%LP:GD)'506;00UCp7h1,K,%tM0YlNEaUT:]+a\8skPCJO?gT#4n&*;Xn)\!aM*nh[5Mj(J#"#HcEJGJVaZ_BQ=YL8ueP8/Hjk44L,)e93B%_DM`Rd-cmMJ^!kh;-)KF6GA)6)4_=qRoW7NF"nKIje36\1J=Uq$q"g^d;JkgS4dk_:8:WhJA8f"uEe]ZJP/Hd.+uia/@h?!!aM80#_J=T4!j>$/@SsqG8N*8ZbQaVX`Xk:RZD"3o(g6\+Rkr/:2pl;/BbRSb'$*s4->U)>)kL:c'[.k2M=O2j3.`_!Se\$lcpV,FqFcLT7R2cV:5:@::jkFur&Do!*iP&^1p+0&Qs9RkkP(GZ5Ll+dCuMU-d8^QAFcToF7n,Y+?S2>2*kE"d66NZJMM7D"N/[VUS%t@B-eJZP7#>B16u!X*PA>d$LbfW^#F`,O`hJ%/*lVB7S$aU,co##Hjc/Xl'VNU:E"Ug-Df<CZ\Hi\gDJ%m/'3IS.D[.h5"ANFN5.QQQM6)[lof^HlH#`^!kV9OE^N-(#VM$0,U8QHak0>oC5P2"iD\ZDJ"g@[l(`1JaL(-TADM,!Y$mFRR!G*j^&&WKb*X*j#Jl^OJ)=7d+mINbMA##R'+5+_]Jg8M=(E$dk*^p;Ei/6cW1(aPE];;5%"UO_G3i?Yk[mqDmJKMPZZ[3D$oV[G-0/u7<b!(7i-E>'Yp&a01S-%ClBtf,_qj?rH]%mf(tF>G\MAg9&ETN1ZSGAXtFjl/)39C1r)V_etm61?OFVNN55H#^]f%K1tf:V+C&K3G!CBB77$5HlQ(c2W1#!J#./;u6@BAUAM3L!G,L>&oKAp%m02@]V_hjR~>endstream
|
||||
endobj
|
||||
32 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1621
|
||||
>>
|
||||
stream
|
||||
Gb!;d=``=U&:WfGfS=(?A0Mij5ltkdc6Tf=9Kd)fX3$9,+;0iNo2&!#Y5@Osi9`m(eB)r?*@:pkhsRtc=R;IW`AajtL+0q()'$A!^c9OR^]Y6'4+ZF[o@5#>k>HOi>XdJ!!sr60Jn!tTi;(;%GecRA8X'o3'I-3R$952PP"@&1/=nj1M"IZG!f1<WYP^KJ?/fpcU':(75IeD@ePB?@!og]CTpj42_/KTo8bK$b+I%0"K38PF%tAtU&+B(c.Do/W-h(2;QuH'A)<VW=]q<"gbIFWI"@qV&,l'JY=nNJ@k:h6pAW?25WsEa.2t9[;$-WLaPZ>G2?pEpfS7shG#j;p?[6sM1X/VnP\1X'1N1C`.>dTH1KeIgE*U]'P[NA-^/d\@@NQ/JCB*lGNcc78:2^`_D0t^qP0alMB"ZS1)TZ;4Y1T+<I)[K<WmQPV-5%VW!g&o`/9'FVfMZeP%9"'Y))C+C<ZZ>?(ZL>g-LMq"ueC!Rqe],YNB0KKTi*6:M>ZZ3Np)0VM`t6hSq=\qV^omnEK\mLe3:U+$OaBI2(S,-W'Q@WH_^m\\r;#XF[pm0T[#]pEhp@8RqcHsG,:h8%R!^nX\Kc,G[,3@)&gO*WIbYN*/(5E#pK=1oY5gM0UZ_?Lh4eP7=_F2F)3]*AOr-D)_e9q0K''BCSD=>JCeOn-*cC>12Ar#ZIe>+job+:;dQ0C8\lH+($U#KE6.TB?NWs6bVU/IV$@AiH`=?4WH8o5`XM&kLRT*p"*("@]&[!8"UNhXPX#H2:&q_a[h`@h]eQ'-5kseDiNp\$*0=i6`peln!5=;4s%;ChJisa6)bnV53L3XE(?^bdI`<TrKT*W.`ndGUW2+\Kp98H6.Rm^o4,!Na\WLn(MmaIqd3U923a1'+0+20Bm$D7<Io&F#k#%g\Sj>iV*SeqmC/j\J$X!LJH1`'2jb-S!?7C+gPH(N^)/?c<^8mu#dl+21O$$8rF*8;`JRXKB<=^sSnA$Y:eri#0s='.1P6d.P:*5^s<P=Lp@pZ-[)'q;G]'Nj_1S1BS'0elhO(FeA+XA&0*;X0iA)-khZCQ73WO8HJu/@8_Q%EBH0dE:YYlLQ8V2W_rDGuPZ):VOU*OdXL;C"gg8Qo2l62EGGR`YH)V3Gr5OC8Q%cfd'gOJ)1nms'^3Ih=\F:,C9LE.L#cKDU0H=!\WS&$<gYg0T*h-G*W$6ECJ\.@2hg`;iIW7OL"+Aqm"'O=CL7,@An*KI6n+&it9hd$(ZO(1!eGGILXhg8`C%,VZg_c-e+?+%JgZi)Lg<s8LcTD9F!74aKhA")E?)0f67,.B"5O`;fX%[I6m=;PQDJ:`dJ+t`!B?0^WCR)TC]^_@j)l/Wc_VPVjk*&d<:#/g``AF>YeaD=JX@j"q.jn\)h?.$TpQp>]$d)E]8dr&0[:t29=;T.pOLo9bA:g#mY\Z7@;H!/$+0h/@0n%f$WIW!#8:(V<]\ALm+;7m2e!R[Cbb1=ptDjFiWIl-,Ln@.oj1_H>7F4?PkbIZA=G28f-X3!K:BafnHuNb)9pl]^$3pg;^P<c\l6/_Mabq_G:SH!$r]A`\6*CH+*d6!GqefdL*K]7GemtmFBKOjY=*eYO<s*`+`J-pWJP%%F2^Ufd>D~>endstream
|
||||
endobj
|
||||
33 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1654
|
||||
>>
|
||||
stream
|
||||
Gb"/'D/\/e&H8h>E<A*Al"XCthP^JrOL/^;/8`hbl_f,=,fWk9MloH.gOB)rYEn5&M2]YqJ_>c0PLSDOp[[f%$7<K$q&RZZN^kdN7tEE4h#b!@%ARu()&Zci_Hc6V%M4PfQHt*V<q1c`Ca@rK`:]NoJX[_>SddTUKQEaf_tuQWHB:b/I:5qpSniZ-#/[A"Ed-o"Mn"_>C7@2clT`%s[Z;6eq8BD;6SkbR$H!Uc#1Z>RCgp;_F'phWK+Acjces")Q<>n=.Y^01.YES4CW8Ir3ZB&M=d3V<_HXR4Z's!LU(:[S#GW)hY)[naiWQN2X7FSJ#1N\OagAX]`*S=@PR*p_8'jbJC*Ui3[ABJ:M.N(r#%O^S%1>>:/_d[=Ql4*keR^fX)pBP(C%KUej7Omk:sW_/&Z8bF)!r/H:ek^JWZCYMSaSfogS35eepq?T5QdMAfUt2sQ]l#3aDA6eL:OZ8`]pSfp!#(I%e@E&(hHS[mje%KG=D(eg$98Wk=6n3/pbuZMuuMZ(3b!@1J0`[+ggh3_r$IU[oD9![][DX`1]A,mYZV></#Uj,<8R.RGLS1LTsMob6F<&D)Qce`%KbEZ<_/1pn@>s_3H"9,+51^'s1ipl`THje13a&'VY-CMCqLi,;gQ&YRh*lA=up0R1s?hJY-8dNPr[8>qJl4Faj=]^sY!Z[$\V^BO=uiNph/X1X##03it9@V68s0NYU;6g/qMt>hMP:a[Qq\ZY"fcVI%RF]]-65DnqI$71r(`cna@S(4#UVOKg/UA%NnbUC8pnar%8lhPc9)j\%Qs4h3a-OCDCDZN]<o!48<tE&W/7R.-Kg0ZIBC=]=m2;lMbuhn@Mh+6(jRG]mK2C\X;-l*O2-R;E/?eF,LA`LJJIS-p@:Fo6#$CjU!-&pE'naFe9nGoMa]IEJ7n.s5t,$4GGO+r/2fn1,2h0<i-O+;0*`S/6W#4$#sYpP3XcX'[%5$!\LA^+>q:AK9N#603EalYZ!!mLRlH="dZX]*"0)mdb<JX3G@)0B,OGBIg;MinA6.:.9[6_H;ALm_Yk8PDpUXEtQ<$)D+AHpZ&>_EI0WFp_J"Y0!Gtp1pmuIAB3A_$q'B%1AGg31.'R3TOcGE/?:SD=<_0e3R'C0VtkQ]@N<H6?2%2\G'Ub;,(6kpN#R?Fej!#p+@P#^=2ui())_dm,W3_!ImNd-ph+bup`gU8kCQa=5u<@P=U'46nHr__'d<k8Uds/_?,X"8M6A&Y=L36K,3<7us23k\&!-'A5!0maW[O)@Vo=raWTQ4fn&:jIhIB4/.dP3J7q_>Um2s[ud?6=T1s!N_$0>V5WZW6]rs+*IjfkkcD<E7gp:,1Nb'JZSR*,UQn.>L[(A_(.1SShp6mPU8ap[5l$>&p!#LGVO*j)`>%[`=CB*m*`)<j_A;o3Aug5@ORrB9QZNNSj45NQs+,;KS+J?sQH/],I:;G7M\?WncoT"MA,Gf$f]kD;[[Y+AaDLfEA_`$llK<];02`;(t;QMo_#1te5k&j8h:4::#+8i-`5a)9j.l84ZKhQCRs=]7@+On[N_BGV3-B(:(gnEfCq4ILu6CCH:PGM03jqT^#?T'04Hj]!XdUdg<^pbKsl"kt*YIZ$\*-eKq((.E_&F`[trI;3kPfYO*o91Yj:FMD5]$bJ&)AH~>endstream
|
||||
endobj
|
||||
xref
|
||||
0 34
|
||||
0000000000 65535 f
|
||||
0000000061 00000 n
|
||||
0000000112 00000 n
|
||||
0000000219 00000 n
|
||||
0000000331 00000 n
|
||||
0000000536 00000 n
|
||||
0000000741 00000 n
|
||||
0000000946 00000 n
|
||||
0000001051 00000 n
|
||||
0000001256 00000 n
|
||||
0000001461 00000 n
|
||||
0000001667 00000 n
|
||||
0000001873 00000 n
|
||||
0000002079 00000 n
|
||||
0000002285 00000 n
|
||||
0000002491 00000 n
|
||||
0000002697 00000 n
|
||||
0000002903 00000 n
|
||||
0000003109 00000 n
|
||||
0000003179 00000 n
|
||||
0000003471 00000 n
|
||||
0000003615 00000 n
|
||||
0000005563 00000 n
|
||||
0000006055 00000 n
|
||||
0000007394 00000 n
|
||||
0000009280 00000 n
|
||||
0000010884 00000 n
|
||||
0000012815 00000 n
|
||||
0000014788 00000 n
|
||||
0000016474 00000 n
|
||||
0000018263 00000 n
|
||||
0000019939 00000 n
|
||||
0000021663 00000 n
|
||||
0000023376 00000 n
|
||||
trailer
|
||||
<<
|
||||
/ID
|
||||
[<30abca0f3b559a6a6508c5a860239030><30abca0f3b559a6a6508c5a860239030>]
|
||||
% ReportLab generated PDF document -- digest (opensource)
|
||||
|
||||
/Info 19 0 R
|
||||
/Root 18 0 R
|
||||
/Size 34
|
||||
>>
|
||||
startxref
|
||||
25122
|
||||
%%EOF
|
||||
478
Les16-RAG-Embeddings/Les16-Slide-Overzicht.md
Normal file
478
Les16-RAG-Embeddings/Les16-Slide-Overzicht.md
Normal file
@@ -0,0 +1,478 @@
|
||||
# Les 15 — RAG + Embeddings
|
||||
## Slide Overzicht (Klas A — 3 uur fysiek, demo-driven)
|
||||
|
||||
**Lesvorm:** Tim demonstreert klassikaal. Studenten kijken. Zelf bouwen = huiswerk.
|
||||
**Demo-app:** Nieuwe PDF Q&A app from scratch
|
||||
**Vervolg op:** Les 14 — Agents
|
||||
**Aansluit op:** Les 16 — MCP servers
|
||||
|
||||
---
|
||||
|
||||
## Slide 1: Title
|
||||
### Les 15 — RAG + Embeddings
|
||||
|
||||
**Visual:**
|
||||
- Background: CREAM
|
||||
- "Les 15" in BLUE
|
||||
- "RAG + Embeddings" in BLACK
|
||||
- Subtitle: "AI laten antwoorden op basis van jouw eigen documenten"
|
||||
|
||||
---
|
||||
|
||||
## Slide 2: Terugblik
|
||||
### Waar staan we?
|
||||
|
||||
**Vorige lessen:**
|
||||
- Les 12: Tool Calling — AI kiest welke functie
|
||||
- Les 13: Agents — autonoom 20-50 stappen
|
||||
- Les 15: Externe APIs + Vercel deploy
|
||||
|
||||
**Het probleem dat we nu oplossen:**
|
||||
|
||||
AI weet veel, maar weet NIET wat in jouw documenten staat:
|
||||
- Interne kennisbank
|
||||
- Klantcontracten
|
||||
- Productdocumentatie
|
||||
- 200-pagina PDF die vandaag binnenkwam
|
||||
|
||||
**Twee opties:**
|
||||
- ❌ Alle 200 pagina's elke vraag meesturen — niet schaalbaar
|
||||
- ✅ **RAG** — alleen de relevante stukjes ophalen per vraag
|
||||
|
||||
**Visual:** Pijl van "alle docs context" naar "RAG → top 5 chunks".
|
||||
|
||||
---
|
||||
|
||||
## Slide 3: Planning
|
||||
### Vandaag — 180 minuten
|
||||
|
||||
| Onderwerp | Duur |
|
||||
|-----------|------|
|
||||
| Welkom + Terugblik | 10 min |
|
||||
| Theorie: Wat is een embedding? | 20 min |
|
||||
| Theorie: RAG pipeline (index + query) | 15 min |
|
||||
| Theorie: pgvector + chunking | 15 min |
|
||||
| **Live Demo 1** — pgvector setup + embed pipeline | 25 min |
|
||||
| **Live Demo 2** — PDF upload + index | 20 min |
|
||||
| **Pauze** | 15 min |
|
||||
| **Live Demo 3** — Query pipeline + AI antwoord | 20 min |
|
||||
| **Live Demo 4** — RAG-tool in een agent | 15 min |
|
||||
| Wanneer RAG wel/niet | 10 min |
|
||||
| Lesopdracht + Huiswerk | 10 min |
|
||||
| Vragen + Afsluiting | 5 min |
|
||||
|
||||
---
|
||||
|
||||
## Slide 4: Wat is een embedding?
|
||||
### Tekst als een punt in de ruimte
|
||||
|
||||
**Het idee:**
|
||||
- Stop een stuk tekst in een embedding-model
|
||||
- Krijg een **vector** terug — array van ~1536 nummers
|
||||
- Vergelijkbare betekenissen = vergelijkbare vectors
|
||||
|
||||
```
|
||||
"De kat zit op de mat" → [0.12, -0.45, 0.88, ...]
|
||||
"Een poes ligt op het tapijt" → [0.14, -0.41, 0.85, ...]
|
||||
"Voetbal in Nederland" → [-0.73, 0.21, -0.32, ...]
|
||||
```
|
||||
|
||||
Eerste twee zijn **dichtbij in vector-space**. Derde is ver weg. Niet omdat woorden overlappen — omdat **betekenis** vergelijkbaar is.
|
||||
|
||||
**Welk model:**
|
||||
- OpenAI `text-embedding-3-small` — 1536 dim, snel + goedkoop
|
||||
- OpenAI `text-embedding-3-large` — 3072 dim, beter maar duurder
|
||||
- Open-source: `nomic-embed-text`, `mxbai-embed-large`
|
||||
|
||||
**Visual:** 2D plot met 3 punten — twee dichtbij, één ver weg.
|
||||
|
||||
---
|
||||
|
||||
## Slide 5: Vector similarity
|
||||
### Hoe meet je 'dichtbij'?
|
||||
|
||||
**Drie metrics:**
|
||||
|
||||
| Metric | Wat | Wanneer |
|
||||
|--------|-----|---------|
|
||||
| **Cosine similarity** | Hoek tussen vectors (–1 tot 1) | Default — werkt op semantic search |
|
||||
| **Dot product** | Sum van producten | Sneller — als vectors genormaliseerd zijn |
|
||||
| **Euclidean distance** | Afstand in ruimte | Zelden voor text — meer voor images |
|
||||
|
||||
**Cosine similarity = 1** → exact gelijke betekenis
|
||||
**Cosine similarity = 0** → ongerelateerd
|
||||
**Cosine similarity = -1** → tegengesteld
|
||||
|
||||
```sql
|
||||
-- In pgvector
|
||||
SELECT * FROM chunks
|
||||
ORDER BY embedding <=> '[0.12, -0.45, ...]'::vector
|
||||
LIMIT 5;
|
||||
```
|
||||
|
||||
`<=>` is cosine distance. Sorteren op `<=>` = dichtstbijzijnde eerst.
|
||||
|
||||
**Visual:** Drie vectors met hoeken — kleine hoek = similar.
|
||||
|
||||
---
|
||||
|
||||
## Slide 6: RAG pipeline
|
||||
### Index time vs query time
|
||||
|
||||
**Index time (eenmalig of bij upload):**
|
||||
|
||||
```
|
||||
PDF / docs
|
||||
↓
|
||||
Parse + chunk (500 tokens per chunk)
|
||||
↓
|
||||
Embed elke chunk
|
||||
↓
|
||||
Store: chunk_text + embedding in pgvector
|
||||
```
|
||||
|
||||
**Query time (elke vraag):**
|
||||
|
||||
```
|
||||
User vraag
|
||||
↓
|
||||
Embed vraag
|
||||
↓
|
||||
Similarity search → top 5 chunks
|
||||
↓
|
||||
Geef chunks als context aan LLM
|
||||
↓
|
||||
LLM antwoordt op basis van context
|
||||
```
|
||||
|
||||
**Belangrijkste inzicht:** AI ziet nooit de hele DB. Alleen 5 relevante chunks per vraag.
|
||||
|
||||
**Visual:** Twee parallelle flow-diagrammen — index vs query.
|
||||
|
||||
---
|
||||
|
||||
## Slide 7: pgvector
|
||||
### Postgres met vector-support
|
||||
|
||||
**Wat is het:**
|
||||
- Postgres extension — voegt `vector` datatype toe
|
||||
- Cosine, euclidean, dot product operators
|
||||
- Schaalbaar tot ~1M+ vectors per tabel
|
||||
- Ingebouwd in Supabase — één click activeren
|
||||
|
||||
**Schema voorbeeld:**
|
||||
|
||||
```sql
|
||||
create extension if not exists vector;
|
||||
|
||||
create table chunks (
|
||||
id bigserial primary key,
|
||||
source text not null,
|
||||
page int,
|
||||
content text not null,
|
||||
embedding vector(1536),
|
||||
created_at timestamp default now()
|
||||
);
|
||||
|
||||
create index on chunks
|
||||
using hnsw (embedding vector_cosine_ops);
|
||||
```
|
||||
|
||||
`hnsw` index = snelle approximate nearest neighbor search. Voor production met >10k rows.
|
||||
|
||||
---
|
||||
|
||||
## Slide 8: Chunking strategieën
|
||||
### Hoe knip je een document op?
|
||||
|
||||
**Drie aanpakken:**
|
||||
|
||||
| Strategie | Hoe | Wanneer |
|
||||
|-----------|-----|---------|
|
||||
| **Fixed size** | 500 tokens per chunk, 50 overlap | Default, simpelste |
|
||||
| **Recursive** | Splits op paragraaf → zin → woord | Behoudt structuur |
|
||||
| **Semantic** | Embed zinnen, group similar | Beste kwaliteit, duurder |
|
||||
|
||||
**Wat is een goede chunk:**
|
||||
- ~200-500 tokens (~1000-2500 chars)
|
||||
- Genoeg context maar niet te groot
|
||||
- **Overlap** (10-15%) zodat info aan grenzen niet verdwijnt
|
||||
|
||||
**Te kleine chunks** → fragmentatie, AI mist context
|
||||
**Te grote chunks** → ruis, irrelevant info verdunt het antwoord
|
||||
|
||||
**Visual:** Document opgeknipt in chunks met overlap.
|
||||
|
||||
---
|
||||
|
||||
## Slide 9: Wat we vandaag bouwen
|
||||
### PDF Q&A from scratch
|
||||
|
||||
**Doel:** App waar je een PDF uploadt en daarna vragen kunt stellen over de inhoud.
|
||||
|
||||
**Stack:**
|
||||
- Next.js 16 + TypeScript + Tailwind
|
||||
- Supabase + pgvector
|
||||
- AI SDK `embedMany` + `generateText`
|
||||
- OpenAI `text-embedding-3-small`
|
||||
- `unpdf` voor PDF parsing
|
||||
|
||||
**Twee endpoints:**
|
||||
|
||||
| Route | Wat | Wanneer |
|
||||
|-------|-----|---------|
|
||||
| `POST /api/index` | PDF upload → chunks → embeddings | Eén keer per document |
|
||||
| `POST /api/ask` | Vraag → embed → search → answer | Per vraag |
|
||||
|
||||
**Bonus:** RAG-tool in een ToolLoopAgent — combineer met Les 13.
|
||||
|
||||
**Visual:** Index + query pipelines uitgelegd in 1 diagram.
|
||||
|
||||
---
|
||||
|
||||
## Slide 10: LIVE DEMO 1 — pgvector + embed pipeline
|
||||
### ~25 min
|
||||
|
||||
**Wat ik laat zien:**
|
||||
1. `pnpm create next-app pdf-qa` + `pnpm add ai @ai-sdk/openai @supabase/supabase-js unpdf`
|
||||
2. **Supabase:** activeer `pgvector` extension (één SQL command)
|
||||
3. Schema: `chunks` tabel met `vector(1536)` kolom + HNSW index
|
||||
4. `lib/embeddings.ts`:
|
||||
```typescript
|
||||
import { embed, embedMany } from "ai";
|
||||
import { openai } from "@ai-sdk/openai";
|
||||
|
||||
const model = openai.textEmbeddingModel("text-embedding-3-small");
|
||||
|
||||
export async function embedText(text: string) {
|
||||
const { embedding } = await embed({ model, value: text });
|
||||
return embedding;
|
||||
}
|
||||
```
|
||||
5. Test: `console.log` een embedding — array van 1536 nummers tussen -1 en 1
|
||||
|
||||
**Visual:** Console-log met vector + Supabase tabel.
|
||||
|
||||
---
|
||||
|
||||
## Slide 11: LIVE DEMO 2 — PDF upload + index
|
||||
### ~20 min
|
||||
|
||||
**Wat ik laat zien:**
|
||||
1. `app/api/index/route.ts` — POST endpoint accepteert file upload
|
||||
2. `unpdf`: extract text per page
|
||||
3. **Chunking**: split text in ~500 char chunks met 50 char overlap
|
||||
4. `embedMany`: batch embed alle chunks (sneller dan loop)
|
||||
5. Insert in Supabase met `embedding` als vector
|
||||
|
||||
```typescript
|
||||
const chunks = chunkText(fullText, 500, 50);
|
||||
const { embeddings } = await embedMany({ model, values: chunks });
|
||||
await supabase.from("chunks").insert(
|
||||
chunks.map((c, i) => ({
|
||||
source: filename,
|
||||
content: c,
|
||||
embedding: embeddings[i],
|
||||
}))
|
||||
);
|
||||
```
|
||||
|
||||
6. Demo: upload Polderfest 2027 line-up PDF — 30 chunks in DB
|
||||
7. Check Supabase tabel — chunks zichtbaar, embedding kolom gevuld
|
||||
|
||||
---
|
||||
|
||||
## Slide 12: Pauze
|
||||
### 15 minuten
|
||||
|
||||
---
|
||||
|
||||
## Slide 13: LIVE DEMO 3 — Query pipeline
|
||||
### ~20 min
|
||||
|
||||
**Wat ik laat zien:**
|
||||
1. `app/api/ask/route.ts` — POST endpoint accepteert vraag
|
||||
2. Embed vraag (zelfde model als index!)
|
||||
3. Supabase RPC of raw SQL: cosine similarity search
|
||||
|
||||
```typescript
|
||||
const { embedding } = await embed({ model, value: question });
|
||||
const { data: chunks } = await supabase.rpc("match_chunks", {
|
||||
query_embedding: embedding,
|
||||
match_count: 5,
|
||||
});
|
||||
|
||||
const context = chunks.map((c) => c.content).join("\n\n");
|
||||
const { text } = await generateText({
|
||||
model: openai("gpt-4o-mini"),
|
||||
prompt: `Beantwoord op basis van deze context:\n\n${context}\n\nVraag: ${question}`,
|
||||
});
|
||||
```
|
||||
|
||||
4. Supabase function `match_chunks` definiëren (SQL):
|
||||
|
||||
```sql
|
||||
create function match_chunks(query_embedding vector(1536), match_count int)
|
||||
returns table (content text, similarity float)
|
||||
language sql as $$
|
||||
select content, 1 - (embedding <=> query_embedding) as similarity
|
||||
from chunks
|
||||
order by embedding <=> query_embedding
|
||||
limit match_count;
|
||||
$$;
|
||||
```
|
||||
|
||||
5. Test: vraag "Wie zijn de headliners?" → AI geeft accuraat antwoord op basis van PDF
|
||||
|
||||
---
|
||||
|
||||
## Slide 14: LIVE DEMO 4 — RAG-tool in een agent
|
||||
### ~15 min
|
||||
|
||||
**Wat ik laat zien:**
|
||||
|
||||
Combineer Les 13 (Agents) + Les 15 (RAG):
|
||||
|
||||
```typescript
|
||||
import { ToolLoopAgent, tool, stepCountIs } from "ai";
|
||||
|
||||
const ragSearch = tool({
|
||||
description: "Zoek in geüploade documenten",
|
||||
inputSchema: z.object({ query: z.string() }),
|
||||
execute: async ({ query }) => {
|
||||
const { embedding } = await embed({ model, value: query });
|
||||
const { data } = await supabase.rpc("match_chunks", {
|
||||
query_embedding: embedding,
|
||||
match_count: 5,
|
||||
});
|
||||
return data;
|
||||
},
|
||||
});
|
||||
|
||||
const docAgent = new ToolLoopAgent({
|
||||
model: openai("gpt-4o-mini"),
|
||||
system: "Je beantwoordt vragen door eerst in documenten te zoeken.",
|
||||
tools: { ragSearch },
|
||||
stopWhen: stepCountIs(10),
|
||||
});
|
||||
```
|
||||
|
||||
**Demo vraag:** "Vergelijk de jazz en rock-headliners op Polderfest" — agent doet 2-3 RAG searches, vergelijkt, antwoordt.
|
||||
|
||||
**Krachtig:** RAG + Agent = AI met geheugen + redenering.
|
||||
|
||||
---
|
||||
|
||||
## Slide 15: Wanneer RAG wel/niet?
|
||||
### Niet alles is RAG
|
||||
|
||||
**Wanneer WEL RAG:**
|
||||
- Veel documenten (>50 pagina's totaal)
|
||||
- Documenten veranderen (klantcontracten, kennisbank)
|
||||
- Semantic search nodig (niet alleen keyword)
|
||||
- Privacy: data blijft in jouw DB
|
||||
|
||||
**Wanneer NIET RAG:**
|
||||
- Klein document (<10 pagina's) — gewoon meesturen
|
||||
- Exacte data (prijzen, IDs) — gebruik tool-calls / DB query
|
||||
- Structured data (tabellen, records) — SQL is beter
|
||||
- 1 keer per dag — overhead niet de moeite
|
||||
|
||||
**Alternatieven:**
|
||||
|
||||
| Probleem | Beter dan RAG |
|
||||
|----------|---------------|
|
||||
| 5-pagina manual | Volledige tekst in system prompt |
|
||||
| 10k product records | SQL tool-calls (Les 12) |
|
||||
| Live data | API tool-call |
|
||||
| Code-base navigatie | Tree-sitter / grep, geen embeddings |
|
||||
|
||||
**Hybrid is vaak best:** semantic search + keyword filter.
|
||||
|
||||
---
|
||||
|
||||
## Slide 16: Lesopdracht + Huiswerk
|
||||
### PDF Q&A app deployen
|
||||
|
||||
**Lesopdracht (in-class, 30 min):**
|
||||
- PDF Q&A app opzetten (volg demo)
|
||||
- pgvector in Supabase actief
|
||||
- 1 PDF indexed
|
||||
- 3 vragen werkend
|
||||
|
||||
**Huiswerk (voor Les 16):**
|
||||
- **A:** Eigen PDF (interesse, studie, hobby) — minimaal 20 pagina's
|
||||
- **B:** UI met chat-interface (`useChat` van Les 12)
|
||||
- **C:** RAG-tool in een ToolLoopAgent
|
||||
- **D:** `RAG.md` met:
|
||||
- Beschrijving van je PDF + chunks count
|
||||
- 5 vragen + antwoorden + welke chunks werden opgehaald
|
||||
- 1 vraag waar RAG het **fout** had — analyse waarom
|
||||
- Eén chunking-strategie geprobeerd + resultaat
|
||||
|
||||
**Bonus:**
|
||||
- Hybrid search (semantic + keyword via Postgres full-text)
|
||||
- Re-ranking met `cohere-rerank` of LLM
|
||||
- Streaming antwoord met source-citations
|
||||
|
||||
---
|
||||
|
||||
## Slide 17: Volgende les + Afsluiting
|
||||
### Vragen?
|
||||
|
||||
**Vandaag gezien:**
|
||||
- Embeddings — tekst als vectors in semantic space
|
||||
- Cosine similarity voor 'dichtbij'
|
||||
- RAG pipeline — index time vs query time
|
||||
- pgvector in Supabase met HNSW index
|
||||
- Chunking strategieën
|
||||
- RAG-tool in een agent
|
||||
|
||||
**Volgende les (Les 16): MCP — Model Context Protocol**
|
||||
- Agent = LLM in een loop met tools
|
||||
- ToolLoopAgent — system, tools, stopWhen, prepareStep
|
||||
- 4 stop-condities: stepCountIs, hasToolCall, isLoopFinished, custom
|
||||
- Combo: RAG-tool in een agent (combo Les 15 + 15)
|
||||
|
||||
**Daarna in deze leerlijn:**
|
||||
- Les 16: MCP — eigen Model Context Protocol server bouwen
|
||||
- Les 17: Externe APIs in diepte (OAuth, webhooks, paid APIs)
|
||||
- Les 18: Supabase Auth + RLS — multi-user apps
|
||||
|
||||
**Vragen? Feedback?**
|
||||
|
||||
---
|
||||
|
||||
## Slide Summary
|
||||
|
||||
| # | Title | Type |
|
||||
|---|-------|------|
|
||||
| 1 | Title | Opening |
|
||||
| 2 | Terugblik | Recap |
|
||||
| 3 | Planning | 180-min |
|
||||
| 4 | Wat is een embedding | Theorie |
|
||||
| 5 | Vector similarity | Theorie |
|
||||
| 6 | RAG pipeline | Theorie |
|
||||
| 7 | pgvector | Theorie |
|
||||
| 8 | Chunking strategieën | Theorie |
|
||||
| 9 | Wat we bouwen | Intro demo |
|
||||
| 10 | **LIVE DEMO 1** — pgvector + embed | Demo |
|
||||
| 11 | **LIVE DEMO 2** — PDF upload + index | Demo |
|
||||
| 12 | Pauze | Break |
|
||||
| 13 | **LIVE DEMO 3** — Query pipeline | Demo |
|
||||
| 14 | **LIVE DEMO 4** — RAG-tool in agent | Demo |
|
||||
| 15 | Wanneer RAG wel/niet | Reflectie |
|
||||
| 16 | Lesopdracht + Huiswerk | Praktijk |
|
||||
| 17 | Afsluiting + Les 16 preview | Closing |
|
||||
|
||||
---
|
||||
|
||||
## Bronnen
|
||||
|
||||
- **AI SDK Embeddings:** https://ai-sdk.dev/docs/ai-sdk-core/embeddings
|
||||
- **OpenAI Embeddings:** https://platform.openai.com/docs/guides/embeddings
|
||||
- **pgvector:** https://github.com/pgvector/pgvector
|
||||
- **Supabase pgvector:** https://supabase.com/docs/guides/database/extensions/pgvector
|
||||
- **unpdf:** https://github.com/unjs/unpdf
|
||||
- **Anthropic — contextual retrieval:** https://www.anthropic.com/news/contextual-retrieval
|
||||
- **LlamaIndex RAG concepts:** https://docs.llamaindex.ai/en/stable/getting_started/concepts/
|
||||
BIN
Les16-RAG-Embeddings/Les16-Slides.pdf
Normal file
BIN
Les16-RAG-Embeddings/Les16-Slides.pdf
Normal file
Binary file not shown.
BIN
Les16-RAG-Embeddings/Les16-Slides.pptx
Normal file
BIN
Les16-RAG-Embeddings/Les16-Slides.pptx
Normal file
Binary file not shown.
BIN
Les16-RAG-Embeddings/lesbestanden/rag-finished.zip
Normal file
BIN
Les16-RAG-Embeddings/lesbestanden/rag-finished.zip
Normal file
Binary file not shown.
BIN
Les16-RAG-Embeddings/lesbestanden/rag-starter.zip
Normal file
BIN
Les16-RAG-Embeddings/lesbestanden/rag-starter.zip
Normal file
Binary file not shown.
Reference in New Issue
Block a user