두 reviewer 의 should-fix 4 건 + nit 5 건 push 전 반영.
## should-fix
- `citation_coverage`: 빈 citations[] 가 `Iterator::all` vacuous-true 로
1.0 새는 거 차단 — `!is_empty() && all(non-empty path)` 로 변경.
또한 `_store: &SqliteStore` dead 인자 시그니처에서 제거 (호출 사이트
+ 테스트 helper 정리).
- `refusal_correctness`: lexical-only run 에서 `answer == None` 인 경우
분모 증가 안 함 (NaN/null 출력) — 자동 fail 처리하던 게 metric 의미를
왜곡함. 새 unit test `refusal_correctness_nan_for_non_rag_run` 추가.
- `groundedness`: `must_contain.is_empty() && forbidden.is_empty()`인
golden 은 분모에서 제외. unconfigured entry 가 free 1.0 받지 않게.
새 unit test `groundedness_skips_unconfigured_goldens` 추가.
- `kb-cli/Cargo.toml` rationale 코멘트 사실 오류 정정 — kb-eval →
kb-app 의존이지 그 반대 아님.
## nits
- `KB_EVAL_GOLDEN` / `DEFAULT_GOLDEN_PATH` 중복 — `metrics::` 의
`pub(crate)` 로 단일화, `runner` 가 import.
- `render_report_md` 의 `{:?}` `ComparisonKind` → 명시적 lowercase
매핑 함수 (`win`/`loss`/`draw`/`regression`) — JSON 직렬화 컨벤션과
통일.
- `extract_chunker_version` `None == None` 매치 silent 위험에 대한
defensive 코멘트.
- `delta_null_when_either_nan` 테스트의 `let mut` suppress hack →
struct update syntax 로 정리.
- `empty_store` test helper + 매번 `mem::forget(tmp)` 죽은 코드 제거.
## 추가 spec doc
`tasks/p5/p5-2-metrics-compare.md` deviations 섹션 4 항목 추가:
- `kb-eval` crate-level `kb-app` dep — P5-1 inheritance, 새 모듈 surface
는 import 안 함.
- `citation_coverage` 약화된 resolver — `document_exists_by_path` 기다리는
중.
- `refusal_correctness` non-RAG 런 NaN.
- `groundedness` no-check golden skip.
## 검증
- `cargo test -p kb-eval` 35/35 (18 unit + 2 loader + 8 integration + 7
runner; 새 3 unit test).
- `cargo clippy --workspace --all-targets -- -D warnings` clean.
- `compare_report_snapshot_matches_fixture` 변경 없이 통과 — 새 동작이
스냅샷 입력 (lexical-only, no must_contain, no should-refuse) 영향 없음.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
State drift after P0–P4 completion + 3 post-merge hotfixes (PR #20
--config across subcommands, PR #24 --config in kb ask, PR #25 RRF
fusion_score normalization). README still framed the project as
"spec frozen, code 0 lines"; phase docs and task specs all carried
status: planned. Sweep:
- README.md: top banner now "P0–P4 done (17/31 tasks) + 3 hotfixes
applied"; command table marks each subcommand's owning phase and
current status (kb ask = ✅ via P4-3, kb eval = ⏳ P5);
phase roadmap table grew a Status column (P0–P4 completed, P5
next, P6–P9 pending); component count bumped 30 → 31 to reflect
P3-5 (app-wiring, post-spec); core decisions table notes the
RRF [0,1] normalization invariant; build+실행 section drops the
"P0 미시작" caveat; new pointers to HOTFIXES.md and SMOKE.md.
- docs/SMOKE.md (new): ~80-line recipe for running the full
pipeline against an isolated /tmp/kb-smoke/ workspace via
--config, without polluting ~/.config/kb/ or
~/.local/share/kb/. Covers fixture seeding, sample config.toml
with the post-merge defaults, doctor → ingest → list →
search × 3 modes → inspect → ask sequence, verification
checklist, and known-behaviour notes (fastembed model
download, RAG response time, --config hard-fail on missing
path).
- tasks/phase-{0..4}-*.md: status frontmatter flipped planned →
completed.
- tasks/p0/, tasks/p1/, tasks/p2/, tasks/p3/, tasks/p4/: same
status flip across all 17 component task specs (1+6+2+5+3).
P5–P9 stay planned.
cargo test --workspace: 319 passed; clippy clean (no source
changes in this commit, just docs + frontmatter).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Bug
config.rag.score_gate default 0.05 was incompatible with hybrid RRF
fusion_score: raw RRF tops out at num_retrievers / (k_rrf + 1) ≈
0.0328 at the default k_rrf=60, so every hybrid `kb ask` tripped
ScoreGate refusal even when the top hit was perfectly aligned across
both retrievers. Symptomatic on the post-P4-3 manual smoke at
/tmp/kb-smoke/ pointed at 192.168.0.47 Ollama:
$ kb ask "Rust ownership 모델의 핵심 규칙은 뭐야?" --mode hybrid
근거 부족. KB에 해당 내용 없음. # top fusion_score = 0.0164
Per-mode score_gate (lexical_score_gate / vector_score_gate /
hybrid_score_gate) was rejected because it forces every consumer
(CLI, eval, TUI) to know which mode picks which threshold. Score
normalization solves it at the source.
## Fix
crates/kb-search/src/hybrid.rs divides every fused score by
2 / (k_rrf + 1), the theoretical RRF maximum with two retrievers
each contributing rank 1. After normalization:
- both retrievers agree on rank 1 → fusion_score = 1.0
- only one retriever finds the chunk → caps near 0.5
- typical mixed ranks → falls between 0 and 0.5
RRF's rank-ordering invariants are preserved (every score divides
by the same positive constant), so sort + tiebreak behaviour is
unchanged. Wire schema label `fusion_score` keeps its slot in
RetrievalDetail; only the magnitude shifts, and only for hybrid
mode (lexical / vector were already in [0, 1]).
Verification: re-ran the four-scenario smoke at /tmp/kb-smoke/ with
default score_gate = 0.05 — all four (Korean→Korean, English→
English, cross-language Korean↔English, out-of-corpus) succeed
with the expected grounded / refusal classification, top
fusion_score now ≈ 0.5.
## Tests
One unit test (rrf_formula_matches_known_value) updated to expect
the normalized value `(1/61 + 1/62) / (2/61) ≈ 0.9919` instead of
the raw `1/61 + 1/62 ≈ 0.0325`. The integration snapshot fixture
crates/kb-search/tests/fixtures/search/hybrid/run-1.json already
used presence checks (fusion_score_positive: true) rather than
absolute values, so it doesn't need regeneration. Workspace 319
tests pass; clippy clean across both feature configs.
## Docs
This commit also adds tasks/HOTFIXES.md as a dated post-merge log
covering this fix and the two earlier --config-flag regressions
(P3-5 hotfix #20 across ingest/search/list/inspect/doctor; P4-3
follow-up #24 for kb ask). Original task specs in tasks/p<N>/
*.md stay frozen as the historical contract; HOTFIXES.md is the
live source of truth for post-merge deltas. Each affected task
spec gets a "Risks/notes" addendum pointing back to HOTFIXES.md
so a reader landing on the spec sees the active behaviour:
- tasks/INDEX.md gains a "Post-merge 핫픽스" section.
- tasks/phase-3-vector-hybrid.md updates the RRF formula text to
show the normalized form.
- tasks/p3/p3-4-hybrid-fusion.md "Behavior contract" RRF bullet
notes the normalization and reason.
- tasks/p3/p3-5-app-wiring.md "Risks/notes" notes the --config
fix.
- tasks/p4/p4-3-rag-pipeline.md "Risks/notes" notes the kb-ask
--config fix and the score_gate-RRF incompatibility (closed by
the normalization in p3-4).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P1–P3 shipped libraries but kb-app facade is still all `bail!("not yet
wired")` stubs, so the CLI is structurally complete but unusable.
Insert p3-5 between P3-4 and P4-1 to swap the facade bodies — ingest,
search, list_docs, inspect_doc, inspect_chunk — into real
compositions of the libraries shipped through P3-4. `ask` stays
stubbed; P4-3 owns it.
After p3-5 merges:
- `kb index` walks a workspace and persists chunks (SQLite +
optionally LanceDB).
- `kb search --mode {lexical,vector,hybrid}` returns real SearchHits
with citations.
- `kb list` / `kb inspect doc|chunk` round-trip from the store.
Updates:
- New task spec at tasks/p3/p3-5-app-wiring.md (depends_on
p1-6/p2-2/p3-2/p3-3/p3-4; unblocks p4-3/p9-1/p9-2/p9-4).
- tasks/INDEX.md bumps P3 component count 4 → 5 and adds the link.
- tasks/phase-3-vector-hybrid.md replaces the speculative
`embed_index` facade signature with the actual frozen kb-app
surface and updates the phase completion checklist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address 8 issues found in spec audit (post PR #2):
1. §refs label: distinguish design vs report sections in p3-1 / p3-2 / p4-2 /
p9-1 / p9-5 contract_sections (e.g., "report §11.2 Ollama" not "§11.2").
2. mock feature gate: gate MockEmbedder (p3-1) and MockLanguageModel (p4-1)
behind `mock` cargo feature, default OFF; add CI symbol-scan as DoD item.
3. Warning type unification: p1-2 frontmatter now emits
`kb_parse_types::Warning` (matches p1-3 / p1-4); drops crate-internal type.
4. p4-3 streaming thread: explicitly single-threaded inside RagPipeline::ask;
collection + sink.send share the calling thread, no race. UI concurrency
is callers responsibility (TUI worker thread pattern in p9-3).
5. p6-2 tesseract version: noted that `tesseract` 0.13 has no stable Rust
`version()` accessor; use TessVersion FFI or shell-out + cache approach.
6. p9-* App struct extensions: introduce `kb_tui::{Library,Search,Ask,Inspect}State`
slots in p9-1 forward-decl form; p9-2/3/4 fill bodies in their own crate
without editing `App`. Parallel-safety contract added.
7. p3-3 cosine score: shift `(sim+1)/2` instead of clamp; preserve ranking
signal between unrelated and opposite vectors. Clamp reserved for NaN.
8. fixtures/ root: p0-1 DoD now creates all fixture subdirs with .gitkeep so
downstream tasks have a stable target path.
PR #1 review left a design-debt note: ParsedBlock landing in kb-core would
(a) force every crate to recompile on parser-internal changes, and
(b) cause namespace pollution when P6/P7/P8 parsers add their own variants.
Resolution: a new thin crate kb-parse-types sits between kb-core and parsers.
Owns ParsedBlock + ParsedPayload + Warning + forward-refs for image/pdf/audio
parser intermediates. Depends on kb-core only (for SourceSpan / Inline).
Updates:
- design §3.7b: add new section defining kb-parse-types
- design §8: add kb-parse-types to module-boundary diagram + forbidden list
- design §3.4 Inline stays in kb-core; kb-parse-types references it (no duplication)
- p0-1 skeleton: workspace + Cargo deps + public surface block
- p1-3 parse-md-blocks: outputs Vec<kb_parse_types::ParsedBlock> directly
- p1-4 normalize: Allowed gains kb-parse-types, drops cross-coupling note
- INDEX + phase-0 epic: list kb-parse-types in P0 deliverables
