docs: Task 0 parity baseline 동결 + gate를 python3/실측 경로로 수정 (output-equality 검증됨)
This commit is contained in:
@@ -54,13 +54,19 @@ while IFS= read -r q; do
|
||||
done < "$BASE/queries.txt"
|
||||
diff "$BASE/ask.jsonl" /tmp/parity/ask.jsonl && echo "ASK IDENTICAL"
|
||||
|
||||
# 4. Chunk byte-identity (proves ingest output unchanged — no re-ingest in Phase 1)
|
||||
sqlite3 "$GATE/data/kebab/kebab.sqlite" \
|
||||
"SELECT chunk_id, text, chunker_version, embedding_version FROM chunks ORDER BY chunk_id" \
|
||||
> /tmp/parity/chunks.tsv
|
||||
# 4. Chunk byte-identity (no sqlite3 CLI on this host → python3 stdlib). DB at
|
||||
# $GATE/data/kebab.sqlite; chunks table has no embedding_version (use policy_hash).
|
||||
python3 - "$GATE/data/kebab.sqlite" /tmp/parity/chunks.tsv <<'PYEOF'
|
||||
import sys, sqlite3
|
||||
c = sqlite3.connect(sys.argv[1])
|
||||
rows = c.execute("SELECT chunk_id, text, chunker_version, policy_hash FROM chunks ORDER BY chunk_id").fetchall()
|
||||
open(sys.argv[2], "w").write("\n".join("\t".join(str(x) for x in r) for r in rows) + "\n")
|
||||
PYEOF
|
||||
diff "$BASE/chunks.tsv" /tmp/parity/chunks.tsv && echo "CHUNKS IDENTICAL"
|
||||
````
|
||||
|
||||
> **Env (this host, verified Task 0):** parity KB `$GATE=/home/user/large_data/out/kebab-parity` (183 docs / 7676 chunks), embedder `snowflake-arctic-embed2` + LLM `gemma3:4b` on **GPU ollama `192.168.0.244:11434`** (lemonade stopped for Phase 1; restore after). `search` AND `ask` deterministic at temp 0 / seed 12345 (2× byte-identical — verified). One gate run ≈ 65s. `sqlite3` CLI NOT installed — python3 stdlib for all DB reads.
|
||||
|
||||
**PASS criteria (all are byte-diffs — output-equality, label-free):**
|
||||
- `SEARCH IDENTICAL` — every query's lexical + hybrid hits byte-identical. No LLM needed; deletions don't touch retrieval. ANY diff is a real regression.
|
||||
- `ASK IDENTICAL` — every query's answer + citations + grounded + refusal + template version identical (volatile timing/run-id stripped via jq). RAG-touching cuts (templates v1/v2, sessions) must leave the default rag-v4 / non-session path identical. (LLM = the KB config's endpoint, self-consistent before/after.)
|
||||
|
||||
@@ -14,6 +14,19 @@ historical contract that was implemented; this file accumulates the
|
||||
deltas so phase 5+ readers can find the live behavior without diffing
|
||||
git history.
|
||||
|
||||
## 2026-06-24 — spine-rewrite Phase 0: 출력-동등성 parity baseline 동결
|
||||
|
||||
척추 재작성/단순화(설계 `2026-06-24-spine-rewrite-simplification-design`)의 per-수정-단위
|
||||
**코어 품질 패리티 게이트** 기준선. eval golden set 이 ground-truth 라벨 부재 → metric 기반
|
||||
대신 **출력 동등성**(`search`/`ask --json` byte-diff + chunk dump diff) 채택(삭제 단계엔 더 정확).
|
||||
|
||||
- parity KB `/home/user/large_data/out/kebab-parity` — corpus = kebab 자체 docs(**183 doc / 7676 chunk**),
|
||||
임베더 `snowflake-arctic-embed2` + LLM `gemma3:4b`, 둘 다 R9700 **GPU ollama(.244)**. baseline 바이너리 = main `cea390d`.
|
||||
- 산출물 `baseline/{queries.txt(7), search.jsonl(14=7×lex/hyb), ask.jsonl(7), chunks.tsv(7676)}`.
|
||||
- **결정성 검증**: `search`·`ask` 모두 temp0/seed12345 에서 2× byte-identical(ollama gemma3 정상 chat-templating). 게이트 1회 ≈ 65s.
|
||||
- 인프라: `sqlite3` CLI 미설치 → python3 stdlib. lemonade 는 Phase 1 동안 stop(opencode/hermes down), 종료 후 복원.
|
||||
- 게이트 정의: `docs/superpowers/plans/2026-06-24-spine-phase0-baseline-and-cuts.md` "Parity Gate".
|
||||
|
||||
## 2026-06-24 — RAG provenance 라벨: `rag-v4` (출처/trust 라벨 + 신뢰도 우선 지시)
|
||||
|
||||
**무엇을 추가했나.** RAG 프롬프트의 각 [근거] 청크 머리에 출처/trust 라벨을
|
||||
|
||||
Reference in New Issue
Block a user