Optimization plan

Primary objective: make Q&A reliably grounded in your markdown docs, with citations, before doing more model scaling.

Next Optimization Plan (RAG-First, Working-First)

Primary objective: make Q&A reliably grounded in your markdown docs, with citations, before doing more model scaling.

Why Responses Still Drift

A ~4M parameter model can assist, but it is not reliable as the main factual answer engine.

Definition Of Done

The system is considered "working" when all are true:

Current Operating Mode (Use Now)

Use grounded extraction as default answer mode:

Deliverables

data/index/chunks.jsonl with metadata: - chunkid, relativepath, startline, endline, sha256, tokenest, text

Build Tasks

Add heading-aware markdown chunking. - 450 token target - 120 token overlap - hard cap 700 tokens - boost docs containing leumas, company, about, overview, who-we-are in path/title. - lexical score - title/path overlap with query terms - freshness bonus (based on modified time metadata). - .\scripts\slm.ps1 ask --model <m --question "<q" --answer-mode extractive --top-k 8 - store top chunks and scores to data/evals/debuglastretrieval.json.

Acceptance

For "What is Leumas Tech?" top chunks include your canonical Leumas docs first.

Tasks

Remove repetitive boilerplate and nav/footer blocks. - nodemodules, dist, build, .git, venv dirs - timestamped npm folders like NPM-2025-12-01T19-43-52-861Z (already implemented).

Gold Set

Create data/evals/qagold.jsonl with at least 50 questions:

Gate

Do not promote a new model unless it beats or matches previous retrieval and answer scores.

Next Session Execution Checklist

Build chunk artifacts and retrieval debug output.