Commit Graph

601 Commits

Author SHA1 Message Date
th-kim0823
de9016fe16 plan(fb-42): bulk multi-query implementation plan
8 tasks: kebab-core types, kebab-app bulk_search_with_config facade
(cap 100 + per-query error policy), CLI --bulk flag + stdin ndjson +
output stream, CLI integration tests, MCP bulk_search tool +
registration + tools_list count bump, MCP integration tests,
capability flag, wire schemas + README + SMOKE + design + SKILL +
status flip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:10:39 +09:00
th-kim0823
35df15df99 spec(fb-42): bulk multi-query design (rerank hint deferred)
- CLI: kebab search --bulk + stdin ndjson → stdout per-query ndjson
- MCP: 신규 kebab__bulk_search tool + JSON envelope (results + summary)
- Sequential for-loop, App instance 재사용 (cache amortize)
- Per-query error policy: continue + per-item error.v1
- Limits: queries.len() <= 100
- Capability flag bulk_search 신규
- Rerank hint 별도 task (fb-39 cross-encoder 설계 후)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:05:27 +09:00
b0becf43b8 Merge pull request 'chore(handoff): sync release roadmap with shipped state' (#133) from chore/sync-handoff into main
Reviewed-on: #133
2026-05-10 10:49:23 +00:00
21ecbb00d4 Merge pull request 'feat(fb-40): fact-grounded answer — rag-v2 prompt template' (#132) from feat/fb-40-fact-grounded-answer into main
Reviewed-on: #132
2026-05-10 10:49:06 +00:00
th-kim0823
8cd21e8342 chore(handoff): sync release roadmap with shipped state
- 0.3.0 batch (fb-26/27/28 + fb-29 deferral) marked cut
- 0.4.0 batch (fb-30 MCP + fb-31 single-file) marked cut
- 0.5.0 batch (fb-32..37) marked cut on 2026-05-10
- 0.6.0 in progress: fb-38 + fb-40 merged today, fb-39 pending
- fb-41/42 reframed as 0.7.0+ candidates

Note: PR #132 (fb-40) merge updates roadmap header in spec status
table (already flipped via fb-40 PR).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 19:46:28 +09:00
th-kim0823
b35f163f56 fix(fb-40): address PR #132 round 1 review
Module doc still pinned "rag-v1" — update to reflect dispatched
template via system_prompt_for (rag-v1 legacy / rag-v2 default).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 19:42:57 +09:00
th-kim0823
600c6182fc docs(fb-40): rag-v2 prompt + README + design + SKILL + INDEX
- README: [rag] prompt_template_version default rag-v2 + V2 강화 3 규칙
- design §7: rag-v2 본문 + V1 legacy note
- SKILL.md: mcp__kebab__ask 응답 행태 변화 안내
- task spec: status open → completed, design + plan 링크
- INDEX: fb-40  머지 (2026-05-10)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 19:37:28 +09:00
th-kim0823
0e8b800b6b test(rag): integration tests for rag-v1/v2/unknown dispatch (fb-40)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 19:18:36 +09:00
th-kim0823
126559ce7a fix(fb-40): update test fixtures for rag-v2 default 2026-05-10 19:15:15 +09:00
th-kim0823
137fc4ee31 feat(config): default prompt_template_version rag-v1 → rag-v2 (fb-40) 2026-05-10 19:04:55 +09:00
th-kim0823
59f01f8185 feat(rag): pipeline reads prompt_template_version via helper (fb-40) 2026-05-10 19:02:39 +09:00
th-kim0823
9f70681b77 feat(rag): SYSTEM_PROMPT_RAG_V2 + system_prompt_for dispatch helper (fb-40) 2026-05-10 19:01:05 +09:00
th-kim0823
6d6eb442be plan(fb-40): fact-grounded answer implementation plan
6 tasks: SYSTEM_PROMPT_RAG_V2 + system_prompt_for helper, pipeline
dispatch wiring, config default flip rag-v1 → rag-v2, test fixture
cleanup, integration tests (rag-v1 / rag-v2 / unknown via
CapturingLm wrapper around MockLanguageModel), docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 18:58:35 +09:00
th-kim0823
28d3250546 spec(fb-40): fact-grounded answer design
- rag-v1 → rag-v2 system prompt with 3 신규 규칙 (verbatim span 인용 자도 /
  학습 지식 동원 금지 / 추측 금지)
- system_prompt_for(version) helper dispatch in pipeline
- config default prompt_template_version "rag-v1" → "rag-v2", V1 legacy
  kept for backwards-compat
- Lever C (pre-LLM gate) already shipped (RefusalReason::ScoreGate),
  out of scope here

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 18:55:05 +09:00
945319ae93 Merge pull request 'feat(fb-38): score semantics — score_kind on search_hit.v1 + RRF formula docs' (#131) from feat/fb-38-score-semantics into main
Reviewed-on: #131
2026-05-10 09:38:24 +00:00
th-kim0823
c864bd007f docs(fb-38): wire schema + README + design + SKILL + INDEX 2026-05-10 18:21:55 +09:00
th-kim0823
67aee9f480 test(cli): integration tests for score_kind on lexical mode (fb-38) 2026-05-10 18:12:14 +09:00
th-kim0823
4440fa6659 fix(fb-38): add score_kind to remaining SearchHit literals
Add missing score_kind field to SearchHit constructors in:
- kebab-tui/tests/search.rs::make_hit()
- kebab-eval/tests/metrics_and_compare.rs::hit()
- kebab-eval/src/metrics.rs::hit()

All test fixtures default to Rrf (hybrid mode), matching the field's
Default impl and the test semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 18:08:29 +09:00
th-kim0823
b51cdb9e8f feat(search/hybrid): fuse hits override score_kind to Rrf (fb-38) 2026-05-10 17:56:56 +09:00
th-kim0823
4e739f3cd8 feat(search): add score_kind to VectorRetriever (Cosine) and hybrid test helpers (Rrf)
This commit unblocks Tasks 3 and 4 of fb-38:
- VectorRetriever::build_hit now labels hits with ScoreKind::Cosine
- Hybrid retriever test helpers (mk_hit functions) label synthetic hits with ScoreKind::Rrf
- Updated lexical snapshot fixture to reflect new score_kind field in output

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 17:54:16 +09:00
th-kim0823
3a621bba0d feat(search/lexical): label hits with ScoreKind::Bm25 (fb-38 task 2)
- Add ScoreKind::Bm25 to LexicalRetriever::build_hit SearchHit construction
- Import ScoreKind from kebab_core in lexical.rs
- Add integration test lexical_retriever_hits_carry_bm25_score_kind to verify all
  hits from LexicalRetriever carry score_kind == ScoreKind::Bm25
- Update lexical snapshot test baseline to include new score_kind field

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 17:54:11 +09:00
th-kim0823
3c605b1a5d feat(core): ScoreKind enum + SearchHit.score_kind (fb-38) 2026-05-10 17:49:02 +09:00
th-kim0823
56f20b7235 plan(fb-38): score semantics implementation plan
7 tasks: kebab-core ScoreKind enum + SearchHit field, lexical Bm25
labeling, vector Cosine, hybrid Rrf + search_with_trace pass-through,
cross-crate SearchHit literal cleanup, CLI integration test, docs
(wire schema + README + design + SKILL + INDEX).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 17:45:57 +09:00
th-kim0823
0359bd9682 spec(fb-38): score semantics design
- search_hit.v1 에 optional score_kind 필드 (rrf | bm25 | cosine)
- LexicalRetriever → Bm25, VectorRetriever → Cosine, HybridRetriever → Rrf
- fb-37 search_with_trace 의 mode-dispatch hits 는 underlying retriever 의
  score_kind 그대로 보존
- README + design §4 + SKILL 에 RRF 수식 전체 + "ranking signal, NOT confidence"
  안내, agent 용 trust threshold 는 nested retrieval.{lexical,vector}_score
- additive minor wire — schema bump 없음

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 17:40:47 +09:00
cf3acfc136 Merge pull request 'chore: bump version 0.4 → 0.5' (#130) from chore/bump-v0.5.0 into main
Reviewed-on: #130
v0.5.0
2026-05-10 08:08:06 +00:00
th-kim0823
668e1174cc chore: bump version 0.4 → 0.5
v0.5.0 batches fb-32 (stale doc indicator) + fb-33 (streaming ask)
+ fb-34 (output budget controls) + fb-35 (verbatim fetch) + fb-36
(search filter args) + fb-37 (trace + stats).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 17:04:51 +09:00
745a75a82b Merge pull request 'feat(fb-37): trace + stats — search debug + KB health surface' (#129) from feat/fb-37-trace-and-stats into main
Reviewed-on: #129
2026-05-10 07:59:56 +00:00
th-kim0823
6a33d08aea fix(fb-37): address PR #129 round 1 review
- doc TraceFusionInput.fusion_score semantics (single-mode vs hybrid)
- comment why total_ms vs stage sum can drift (millis truncation)
- TODO marker on TUI trace popup filter passthrough

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:26:34 +09:00
th-kim0823
a40593590b docs(fb-37): wire schema + README + SMOKE + INDEX + SKILL 2026-05-10 14:13:47 +09:00
th-kim0823
5687cbc0e2 feat(tui): search pane t-key opens TracePopup (fb-37) 2026-05-10 13:39:11 +09:00
th-kim0823
653e432a30 feat(mcp): kebab__search trace input + output mirror (fb-37)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 13:32:30 +09:00
th-kim0823
f7e2072d66 test(cli): integration tests for --trace + schema breakdowns (fb-37)
Also fixes App::search_with_opts trace branch to use NoopRetriever
for SearchMode::Lexical, removing the embeddings requirement when
the user only wants lexical-mode trace.
2026-05-10 13:21:33 +09:00
th-kim0823
72c227af23 feat(cli): kebab search --trace flag + wire trace + pretty print (fb-37)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 13:08:48 +09:00
th-kim0823
69037c313a feat(app): SearchResponse.trace + opts.trace threading (fb-37)
Adds the `trace: Option<SearchTrace>` field to `SearchResponse` and
threads `SearchOpts.trace` through `App::search_with_opts`. When the
caller sets `opts.trace = true` the path bypasses the LRU search cache
and runs through `HybridRetriever::search_with_trace`, which dispatches
all 3 SearchModes internally; this means `--trace` requires embeddings
(same constraint as `--mode hybrid`). The non-trace path keeps its
exact prior behavior with `trace: None` stamped on the response.

Picked up Task 1 / Task 3 follow-ups in the same commit so the
workspace compiles: SearchOpts struct-literals in kebab-cli/main.rs +
kebab-mcp/tools/search.rs default the new `trace` field to false, and
the schema-wrapper test in kebab-cli/wire.rs fills the new
media_breakdown / lang_breakdown / index_bytes / stale_doc_count fields
on Stats with `Default::default()`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 13:01:18 +09:00
th-kim0823
6a067e3ab1 feat(search): HybridRetriever::search_with_trace (fb-37) 2026-05-10 12:38:53 +09:00
th-kim0823
231d80e82d feat(stats): media/lang/bytes/stale fields on schema.v1.stats (fb-37)
Extends CountSummary with media_breakdown, lang_breakdown, stale_doc_count
fields populated via stats_ext::breakdowns(). Adds count_summary_with_threshold
for callers that need real stale counts. Mirrors all new fields onto the
wire-bound Stats struct in kebab-app::schema with #[serde(default)] for
backwards-compat. Also fixes search_budget_integration.rs for the trace field
added to SearchOpts in Task 1.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 12:34:57 +09:00
th-kim0823
69c6e23432 feat(store): breakdowns + index_bytes helpers (fb-37)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 12:24:43 +09:00
th-kim0823
1e943f21dc feat(core): SearchTrace + IndexBytes types + SearchOpts.trace (fb-37)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 12:17:04 +09:00
th-kim0823
fb31befef1 plan(fb-37): trace + stats implementation plan
10 tasks: kebab-core types, store breakdowns/index_bytes helpers,
extended CountSummary + Stats wire mirror, HybridRetriever
search_with_trace, App SearchResponse.trace threading, CLI --trace
flag, integration tests, MCP SearchInput.trace, TUI TracePopup,
docs (wire schema + README + SMOKE + INDEX + SKILL).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 12:14:26 +09:00
th-kim0823
5f6b2fa259 spec(fb-37): trace + stats design
- search --trace boolean flag, additive optional `trace` field on search_response.v1
- HybridRetriever search_with_trace returns (hits, SearchTrace) — lex/vec/rrf_inputs + per-stage timing
- cache bypass when --trace (debug intent)
- schema.v1.stats extended with media_breakdown / lang_breakdown / index_bytes / stale_doc_count
- TUI search pane `t` keystroke opens TracePopup
- additive minor wire — no schema bump

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 12:05:31 +09:00
a0497d9c53 Merge pull request 'chore: sync Cargo.lock for kebab-mcp time dep (fb-36)' (#128) from chore/sync-cargo-lock-fb36 into main
Reviewed-on: #128
2026-05-10 02:09:03 +00:00
th-kim0823
b221686133 chore: sync Cargo.lock for kebab-mcp time dep (fb-36)
PR #127 added time = { workspace = true } to kebab-mcp/Cargo.toml
but Cargo.lock entry was not regenerated before merge. cargo build
on main locally regenerates the +time line under kebab-mcp.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 11:04:17 +09:00
a72c6f307c Merge pull request 'feat(fb-36): search filter args (--media / --ingested-after / --doc-id + 4 existing)' (#127) from feat/fb-36-search-filters into main
Reviewed-on: #127
2026-05-10 02:02:24 +00:00
th-kim0823
84287d0ef6 fix(fb-36): address PR #127 round 1 review
- ingested_after: convert OffsetDateTime to UTC before formatting
  so non-Z offsets compare correctly against UTC TEXT storage
  (lexical.rs + filters.rs)
- README: --tag is repeatable-only, not csv (only --media is csv)
- test(cli): add multi-value --tag OR-within IN-list coverage
- test(store): add UTC-offset regression test for ingested_after
- mcp: use ERROR_V1_ID const instead of hardcoded "error.v1"

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 04:47:55 +09:00
th-kim0823
6e7446861b docs(fb-36): README + SMOKE + INDEX + skill notes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 04:26:27 +09:00
th-kim0823
b06f4654e7 feat(mcp): kebab__search filter inputs (fb-36)
7 new optional inputs on SearchInput: tags, lang, path_glob,
trust_min, media, ingested_after, doc_id. Validation surfaces as
error.v1 code = invalid_input via StructuredError. Dispatch builds
SearchFilters from the inputs and forwards through the existing
search_with_opts_with_config facade.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 04:11:27 +09:00
th-kim0823
4e0379c04f test(cli): wire_search_filters — lexical-only integration tests (fb-36)
Cover: --doc-id scoping, --ingested-after validation error,
--media md alias, --tag repeatable + frontmatter parsing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 04:06:21 +09:00
th-kim0823
6a18847892 feat(cli): kebab search filter flags (fb-36)
7 new flags: --tag (repeatable), --lang, --path-glob,
--trust-min (value_enum), --media (csv with `md` alias),
--ingested-after (RFC3339; config_invalid on parse fail),
--doc-id. Dispatch translates clap values into SearchFilters
and propagates structured errors through the existing
StructuredError wrapper from fb-34.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 03:57:55 +09:00
th-kim0823
c6cc1e2bfe feat(search/vector): media / ingested_after / doc_id filters (fb-36)
filter_chunks helper in kebab-store-sqlite extended with the same 3
WHERE clauses as lexical. Vector still over-fetches k*2 then
post-filters via SqliteStore::filter_chunks; small k can return < k
hits when filters drop a lot — agent is expected to widen k or
paginate. AND combinator with existing filters.

- kebab-store-sqlite/src/filters.rs: media IN-list subquery, ingested_after
  lexicographic >= compare, doc_id equality; mirrors lexical SQL arms
- 3 direct unit tests (filter_chunks_media_type/ingested_after/doc_id)
  that run without AVX/Lance
- common/mod.rs: insert_doc / insert_doc_with_media / run_vector_search
  helpers on HybridEnv for integration-test use
- hybrid.rs: 2 new #[ignore = "requires AVX..."] integration tests
  (vector_filter_by_media, vector_filter_by_doc_id)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 03:50:56 +09:00
th-kim0823
86475e5ba2 fix(search/lexical): use std::iter::repeat_n (clippy)
Per code review on 2c80e2a. manual-repeat-n lint triggers
for Rust 1.94+ when repeat().take() can be expressed as
repeat_n directly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 03:43:51 +09:00