docs: Task 0 parity baseline 동결 + gate를 python3/실측 경로로 수정 (output-equality 검증됨)

This commit is contained in:
2026-06-24 09:00:38 +00:00
parent b04b865c73
commit e1edb9224b
2 changed files with 23 additions and 4 deletions

View File

@@ -54,13 +54,19 @@ while IFS= read -r q; do
done < "$BASE/queries.txt"
diff "$BASE/ask.jsonl" /tmp/parity/ask.jsonl && echo "ASK IDENTICAL"
# 4. Chunk byte-identity (proves ingest output unchanged — no re-ingest in Phase 1)
sqlite3 "$GATE/data/kebab/kebab.sqlite" \
"SELECT chunk_id, text, chunker_version, embedding_version FROM chunks ORDER BY chunk_id" \
> /tmp/parity/chunks.tsv
# 4. Chunk byte-identity (no sqlite3 CLI on this host → python3 stdlib). DB at
# $GATE/data/kebab.sqlite; chunks table has no embedding_version (use policy_hash).
python3 - "$GATE/data/kebab.sqlite" /tmp/parity/chunks.tsv <<'PYEOF'
import sys, sqlite3
c = sqlite3.connect(sys.argv[1])
rows = c.execute("SELECT chunk_id, text, chunker_version, policy_hash FROM chunks ORDER BY chunk_id").fetchall()
open(sys.argv[2], "w").write("\n".join("\t".join(str(x) for x in r) for r in rows) + "\n")
PYEOF
diff "$BASE/chunks.tsv" /tmp/parity/chunks.tsv && echo "CHUNKS IDENTICAL"
````
> **Env (this host, verified Task 0):** parity KB `$GATE=/home/user/large_data/out/kebab-parity` (183 docs / 7676 chunks), embedder `snowflake-arctic-embed2` + LLM `gemma3:4b` on **GPU ollama `192.168.0.244:11434`** (lemonade stopped for Phase 1; restore after). `search` AND `ask` deterministic at temp 0 / seed 12345 (2× byte-identical — verified). One gate run ≈ 65s. `sqlite3` CLI NOT installed — python3 stdlib for all DB reads.
**PASS criteria (all are byte-diffs — output-equality, label-free):**
- `SEARCH IDENTICAL` — every query's lexical + hybrid hits byte-identical. No LLM needed; deletions don't touch retrieval. ANY diff is a real regression.
- `ASK IDENTICAL` — every query's answer + citations + grounded + refusal + template version identical (volatile timing/run-id stripped via jq). RAG-touching cuts (templates v1/v2, sessions) must leave the default rag-v4 / non-session path identical. (LLM = the KB config's endpoint, self-consistent before/after.)

View File

@@ -14,6 +14,19 @@ historical contract that was implemented; this file accumulates the
deltas so phase 5+ readers can find the live behavior without diffing
git history.
## 2026-06-24 — spine-rewrite Phase 0: 출력-동등성 parity baseline 동결
척추 재작성/단순화(설계 `2026-06-24-spine-rewrite-simplification-design`)의 per-수정-단위
**코어 품질 패리티 게이트** 기준선. eval golden set 이 ground-truth 라벨 부재 → metric 기반
대신 **출력 동등성**(`search`/`ask --json` byte-diff + chunk dump diff) 채택(삭제 단계엔 더 정확).
- parity KB `/home/user/large_data/out/kebab-parity` — corpus = kebab 자체 docs(**183 doc / 7676 chunk**),
임베더 `snowflake-arctic-embed2` + LLM `gemma3:4b`, 둘 다 R9700 **GPU ollama(.244)**. baseline 바이너리 = main `cea390d`.
- 산출물 `baseline/{queries.txt(7), search.jsonl(14=7×lex/hyb), ask.jsonl(7), chunks.tsv(7676)}`.
- **결정성 검증**: `search`·`ask` 모두 temp0/seed12345 에서 2× byte-identical(ollama gemma3 정상 chat-templating). 게이트 1회 ≈ 65s.
- 인프라: `sqlite3` CLI 미설치 → python3 stdlib. lemonade 는 Phase 1 동안 stop(opencode/hermes down), 종료 후 복원.
- 게이트 정의: `docs/superpowers/plans/2026-06-24-spine-phase0-baseline-and-cuts.md` "Parity Gate".
## 2026-06-24 — RAG provenance 라벨: `rag-v4` (출처/trust 라벨 + 신뢰도 우선 지시)
**무엇을 추가했나.** RAG 프롬프트의 각 [근거] 청크 머리에 출처/trust 라벨을