Commit Graph

25 Commits

Author SHA1 Message Date
17c48a0ee6 feat(cli): fb-41 PR-4 — CLI --multi-hop flag + answer.v1 / error.v1 wire 확장
fb-41 multi-hop RAG 의 **PR-4** (PR-3b-ii 의 ScriptedLm + tests 위에서
user-facing CLI surface + JSON Schema 확장). PR-3b-i / PR-3b-ii 의 multi-hop
pipeline 을 `kebab ask --multi-hop` 으로 사용자에게 노출.

설계: docs/superpowers/specs/2026-05-25-p9-fb-41-multi-hop-rag-design.md
계획: docs/superpowers/plans/2026-05-25-p9-fb-41-multi-hop-rag.md (PR-4 단락)

## CLI surface

- `kebab ask --multi-hop <query>` — 새 flag (default false). `AskOpts.multi_hop`
  로 전달, stream + non-stream 두 callsite 모두 갱신.
- `--show-citations` / `--hide-citations` / `--stream` / `--session` 등 기존
  flag 와 orthogonal.
- `--json` 모드에서 `Answer.hops` 배열이 multi-hop happy path / refusal-with-
  partial-trace 양쪽 경로에서 노출됨 (PR-3b-i + PR-3b-ii 의 wiring).

## Wire schema 확장

- `docs/wire-schema/v1/answer.schema.json`:
  - 신규 `hops: array | null` 필드 (optional, additive). `HopRecord` 의
    `$defs` 추가 — `iter` / `kind` (decompose|decide|synthesize) /
    `sub_queries` / `context_chunks_added` / `forced_stop` / `llm_call_ms`
    6 필드 + per-field doc.
  - `refusal_reason` 필드를 `anyOf [enum, null]` 로 명시 — 6 variant
    (`score_gate`, `llm_self_judge`, `no_index`, `no_chunks`,
    `llm_stream_aborted`, `multi_hop_decompose_failed`). 이전 schema 는
    `type: string|null` 만 명시 → enum 명시는 agent / consumer 의 strict
    validate 강화 (additive — 기존 producer 값 모두 enum 안).
  - `$id` / `schema_version` 변경 없음 — additive minor.
- `docs/wire-schema/v1/error.schema.json`:
  - `code` enum 에 `multi_hop_decompose_failed` 추가. **이는 forward-looking
    enum extension** — 현재 RefusalReason 은 `Answer.refusal_reason` (stdout)
    으로만 노출되고 `error.v1` (stderr) 경로 안 거침. 미래 PR 에서 fatal
    promotion 정책 결정 시 trigger 가능하도록 enum 만 미리 reserve.
  - details.description 의 per-code 안내에 `multi_hop_decompose_failed: {}`
    note 추가 — reserved 상태 명시.

## Tests

- `crates/kebab-cli/tests/wire_ask_multi_hop.rs` 신규 (4 Ollama-free pins):
  - `cli_ask_help_advertises_multi_hop_flag`: clap-level smoke, `kebab ask
    --help` 출력에 `--multi-hop` 등장 확인.
  - `answer_schema_declares_hops_property_with_hop_record_defs`: `hops`
    property 존재 + `$defs.HopRecord` 의 `kind` enum 3 variant
    (decompose/decide/synthesize) 회귀 핀.
  - `answer_schema_refusal_reason_enum_includes_multi_hop_decompose_failed`:
    6 variant 모두 enum 에 존재 — 기존 5 도 함께 핀 (회귀 방지).
  - `error_schema_code_enum_includes_multi_hop_decompose_failed`: 신규
    code enum 확장 + 기존 code (config_invalid / not_indexed / ...) 보존 핀.

End-to-end multi-hop ask 의 live Ollama 검증은 후속 `#[ignore]` test 로
(같은 `wire_ask_stale.rs` 패턴). PR-4 의 범위 = clap + schema 정합성 만.

## 변경 없음

- `crates/kebab-app/src/error_wire.rs` — plan 의 "error_wire 매핑" 항목은
  현재 RefusalReason 가 `Answer.refusal_reason` 로만 노출 (anyhow chain 안
  거침) 라 trigger 가 없음. enum reservation 만으로 충분, 매핑 코드는 dead
  code 회피. 향후 fatal-promotion 정책 (refusal → error.v1) 결정 시 PR-4b
  로 split.
- `prompt_template_version` — `rag-multi-hop-v1` 그대로.
- TUI / MCP surface — PR-5 / PR-6 에서.

## 검증

- `cargo test -p kebab-cli -j 1` — 모든 test 통과 (신규 wire_ask_multi_hop 4 +
  기존 ask / search / schema / ingest / mcp / reset 등 모두).
- `cargo clippy -p kebab-cli --all-targets -j 1 -- -D warnings` clean.
- 단일 crate 직렬 build (16 GB RAM 제약).

## 다음 PR

- PR-5: MCP `ask` tool 의 `multi_hop: bool` argument + `integrations/claude-
  code/kebab/SKILL.md` 의 ask 절 갱신.
- PR-6: TUI Ask 패널 multi-hop toggle (F2 / Ctrl-T) + hop trace render.
- v0.18.0 cut (PR-6 머지 후): `Cargo.toml` 0.17.2 → 0.18.0 + HANDOFF /
  HOTFIXES / INDEX 갱신 + gitea-release.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 08:45:01 +00:00
0def913abd feat(v0.17.0/PR-C): code_lang_chunk_breakdown additive wire field
closure of HOTFIXES 2026-05-22 "code_lang_breakdown chunk granularity"
LOW. Chunk-level companion of the existing doc-count metric.

- crates/kebab-store-sqlite/src/store.rs: code_lang_chunk_breakdown()
  method. chunks INNER JOIN documents → COUNT(c.chunk_id) GROUP BY
  metadata_json.code_lang, NULL skipped. BTreeMap<String, u32>.
  + lib unit test code_lang_chunk_breakdown_counts_chunks_not_docs
  (1 rust doc + 3 chunks → rust=3 chunks vs rust=1 doc).
- crates/kebab-app/src/schema.rs: Stats.code_lang_chunk_breakdown
  additive field + collect_stats builder. tests_stats_ext 의
  stats_includes_code_lang_and_repo_breakdown_fields 가 신규 필드도
  검증.
- docs/wire-schema/v1/schema.schema.json: 신규 additive 필드
  명세 + 기존 code_lang_breakdown / repo_breakdown description
  정정 ("code chunk count" → "doc count", Gemini round 2 권고).
- tasks/HOTFIXES.md: 2026-05-24 PR-C closure entry.

wire additive, schema_version bump 불필요. v0.16.x 호출 호환.
cargo test --workspace --no-fail-fast -j 1 + clippy 통과.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 20:35:01 +00:00
6ac7fea7b9 feat(v0.17.0/A5): trigram-aware build_match_string + SearchResponse.hint
PR-A 본체. plan Task A4 Step 1c + A5.

- lexical.rs::build_match_string 재설계: whole-phrase + token-AND
  OR-combined, 3자 미만 토큰 drop, 후보 없음 시 None (빈 MATCH
  회피). raw single-quote mode 유지.
- SearchResponse.hint additive — empty result + trimmed < 3 chars
  + non-raw 케이스에 short_query_hint helper 가 set.
- CLI 'kebab search' 가 [hint] stderr 한 줄 (text mode).
- TUI SearchState.short_query_hint + poll_worker stale-aware set
  + fire_search/mark_input_changed reset + dynamic_status 표시.
- docs/wire-schema/v1/search_response.schema.json hint additive.
- 신규 unit tests (lexical 9 PASS, 기존 2 expectation 갱신) +
  통합 회귀 (search_korean: multi_token + mixed, 3 PASS) +
  BM25 snapshot regen (trigram token stream).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 11:54:25 +00:00
749c6ae240 docs(dogfood): sync reset_report schema + README for --orphans-only (PR #149 review)
Round 1 review found 2 doc gaps:
- docs/wire-schema/v1/reset_report.schema.json: 'orphans_only' missing
  from scope enum; orphans_purged/purged_paths properties absent
- README: --orphans-only not listed in the reset prose

Schema additions are additive minor (default values keep back-compat).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 07:47:44 +00:00
th-kim0823
7bbd2c0cbf docs(p10-1a-1): wire schema + frozen design + README/HANDOFF/SMOKE + task index
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 17:41:26 +09:00
th-kim0823
441f1192ee docs(fb-42): wire schema + README + SMOKE + design + SKILL + INDEX
- Add bulk_search_item.v1 + bulk_search_response.v1 wire schemas
- Register both in WIRE_SCHEMAS const
- README: --bulk flag mention + MCP tool list 7→8 (bulk_search)
- SMOKE: bulk multi-query walkthrough (CLI + MCP equivalent)
- Design §2.2: Bulk multi-query (fb-42) subsection (additive minor)
- SKILL: mcp__kebab__bulk_search section + tool table row
- Task spec status open→completed, banner replaced
- INDEX: fb-42 row 머지 (rerank hint deferred)
- Fix: missed Capabilities {bulk_search} in cli wire.rs test (Task 7 leftover)
- Fix: missed tools.len() 7→8 in cli_mcp_smoke (Task 5 leftover)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:07:36 +09:00
th-kim0823
e8da415624 feat(schema): bulk_search capability flag (fb-42)
- Capabilities.bulk_search: true (snapshot)
- schema.v1 wire required list updated

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:49:09 +09:00
th-kim0823
c864bd007f docs(fb-38): wire schema + README + design + SKILL + INDEX 2026-05-10 18:21:55 +09:00
th-kim0823
a40593590b docs(fb-37): wire schema + README + SMOKE + INDEX + SKILL 2026-05-10 14:13:47 +09:00
th-kim0823
b86b763dfb fix(fb-35): address PR #126 round 2 review
- wire schema: relax effective_end.minimum 1 → 0 + expand
  description to cover line-clamp + out-of-range sentinel
  (panic-fix R1 emits Some(0) when line_start=1 and range is
  beyond doc end — schema must accept it)
- tests: tighten first-chunk-target boundary test to assert ≤ 2
  total neighbors (3-chunk doc, N=2). Strict "first chunk →
  context_before empty" not assertable until chunks.ordinal
  column lands (R1 #9 architectural caveat)
- store: trim contradiction in list_chunk_ids_for_doc warning
  comment — drop "good enough for sequentially chunked
  markdown" phrase that conflicts with "hash sort dominates"
  paragraph above

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 00:55:29 +09:00
th-kim0823
75eeae3933 feat(wire): fetch_result.v1 schema (fb-35)
Discriminated by kind (chunk / doc / span). Per-kind required
fields enforced by description prose at v1 stub stage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 23:36:19 +09:00
th-kim0823
e084b306e5 fix(fb-34): align next_cursor semantics with docs (PR #125 round 2)
Previous round-1 fix dropped the speculative cursor branch on
the truncated path, leaving a contradiction with the docs:
- snippet-only shrunk → cursor emitted (returned == k_effective)
- k-popped → cursor null (returned < k_effective)
But docs promised the opposite.

R2 resolution: emit cursor whenever more hits may be reachable
(either retriever filled the page OR budget popped hits — the
popped ones remain fetchable from offset+returned). Drop the
artificial "widen vs paginate" copy; truncated and next_cursor
are now independent signals — caller may do either or both.

Updates: app.rs::search_with_opts logic + SearchResponse doc +
schema description + SKILL.md two bullets + max_tokens=0 test
asserts cursor IS emitted on k-pop case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 21:07:04 +09:00
th-kim0823
f485608108 fix(fb-34): address PR #125 round 1 review
- error_wire: StructuredError wrapper preserves ErrorV1 through
  anyhow → classify pipeline. Adds downcast short-circuit so
  cursor::decode's typed code = "stale_cursor" reaches the wire
  instead of being string-formatted to code = "generic".
- app: search_with_opts now wraps cursor::decode error in
  StructuredError instead of anyhow! string format.
- test: error_wire pins both negative (bare anyhow → not
  stale_cursor) AND positive (StructuredError → stale_cursor)
  invariants. CLI integration test runs end-to-end and asserts
  error.v1.code on stderr.
- app: next_cursor only emitted on full-page (k-pop) path; drop
  speculative emit on snippet-only truncation that would point at
  a different page than the agent expected.
- cursor: differentiate malformed-base64 / malformed-payload /
  revision-mismatch error messages; all keep code = stale_cursor.
- test: cursor_rejected fixture uses .expect() to fail loud on
  cursor non-emission instead of silent skip.
- test: max_tokens=0 → 1-hit floor + truncated=true.
- docs: SKILL.md + schema description distinguish snippet-shrink
  (widen) vs k-pop (paginate) truncated cases. HOTFIXES notes
  --no-cache semantic shift (cached path + clear vs uncached path).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 20:49:27 +09:00
th-kim0823
f25ad31741 feat(wire): search_response.v1 schema (fb-34)
Wrapper around search_hit.v1[] with next_cursor + truncated.
Wire breaking — agent that parses bare array must adapt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 18:00:58 +09:00
th-kim0823
a082b78f8e fix(fb-33): address PR #124 round 1 review
- pipeline: refresh module docstring step 5 to reflect new cancel
  semantics (RetrievalDone/Token/Final + LlmStreamAborted)
- wire schema: spell out refusal-path behavior in answer_event.v1
  description (only retrieval_done emitted; no final)
- test: factual comment on relax_score_gate-using test corrected
- test: new Ollama-gated stream_score_gate_refusal_emits_only_retrieval_done
- test: new ask_emits_no_final_when_cancelled_mid_stream pinning
  the no-Final invariant on cancel
- pipeline: large_enum_variant comment broadened to acknowledge
  RetrievalDone.hits as the dominant per-emit cost
- HOTFIXES: log AskOpts.stream_sink internal API break per spec
  contract policy

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 15:46:04 +09:00
th-kim0823
e8caf2a57e feat(wire): answer_event.v1 schema (fb-33)
Discriminated ndjson event for `kebab ask --stream`. Mirrors
the ingest_progress.v1 pattern (stderr stream + stdout final
answer.v1 for backwards compat).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 14:58:49 +09:00
th-kim0823
cc41adabb5 feat(wire): search_hit.v1 + citation.v1 require indexed_at + stale (fb-32)
Additive minor — schema_version unchanged. Existing v1 consumers
that ignore unknown fields stay compatible; consumers that validate
strictly will reject pre-fb-32 payloads, which matches the wire
contract escape hatch (recipient version >= producer required).

Cross-task placeholders: kebab-eval / kebab-tui synthetic test
fixtures pin UNIX_EPOCH + stale=false (same pattern as
hybrid.rs / vector.rs). These don't exercise staleness — Task 11
adds dedicated TUI staleness rendering tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 02:17:15 +09:00
th-kim0823
c25f4f89e3 📝 docs(wire-schema): schema.v1 + error.v1 JSON Schema (fb-27)
schema.v1: full introspection report shape with required fields for
wire / capabilities / models / stats. capabilities object enumerates
all 10 flag names (current 6 true + future 4 false) as required keys.

error.v1: 7-code enum + permissive details object. Real emitted
details shapes documented in description (per-code context varies and
some fields are interim until IoFailure / OpTimeout typed signals
land in follow-up).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 12:35:15 +09:00
693f5582f0 feat(kebab-core, kebab-app): p9-fb-25 task 4 — IngestReport.skipped_by_extension + wire schema additive
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:06:34 +00:00
aa2a6ea7fc feat(kebab-core): p9-fb-23 task 1 — IngestItemKind::Unchanged + IngestReport.unchanged
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 17:43:52 +00:00
ecb85651ea spec(p9-fb-15): RAG multi-turn 정책 + answer.v1 conversation_id/turn_index
도그푸딩 후 추가된 ask multi-turn (꼬리 물기) surface 를 frozen design
+ wire schema 에 명시. p9-fb-15 (RAG core) + p9-fb-16 (TUI UI) +
p9-fb-17 (V004 chat sessions) + p9-fb-18 (CLI session/repl) 의 spec
PR — impl PR 들이 이어진다.

변경:
- §2.3 Answer wire schema: conversation_id (String?) + turn_index
  (u32?) 두 optional 필드. 기존 single-shot 소비자 (외부 wrapper)
  영향 없음 — 두 필드 모두 optional.
- §3.8 RAG types:
  - Answer struct 에 conversation_id / turn_index field 추가.
  - Turn struct 신설 (history 가 prompt 에 들어갈 때 한 turn).
- §3.8 \"Multi-turn behaviour\" 신설 절:
  - kebab-rag::ask vs ask_with_history 두 entry.
  - prompt 빌드 priority: system+question (필수) → retrieved chunks
    (k 줄여 fit) → history (newest 우선, oldest drop).
  - retrieval query expansion (직전 answer 첫 200자 concat).
  - Aborted vs Completed semantics — ask 는 single-shot 이라 cancel
    시 partial token + grounded=false + LlmStreamAborted refusal
    (variant 추가는 p9-fb-15 impl 가 함께).
- docs/wire-schema/v1/answer.schema.json: 두 필드 추가 +
  created_at 에 format: date-time (sibling ingest_progress.v1 와
  일관).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:09:34 +00:00
9aa7459e87 review(회차1): nit 3건 반영
- §10 long-running 절 끝 빈 줄 3 → 1 (다른 절 사이 일관)
- wire schema + §2.4a 예제 JSON: kind_result → result (top-level
  kind 와의 모호성 제거; ingest_report.v1.items[].kind 와 짝)
- wire schema 의 ts 필드: format: \"date-time\" 추가 (RFC 3339
  자동 검증, wrapper 가 다른 format emit 시 즉시 잡힘)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 19:18:32 +00:00
5ef8598e5c spec(p9-fb-01..03): ingest progress events + cancellation in §2.4a / §10
도그푸딩 후 추가된 long-running 작업 진행 표시 + cancel 정책을 frozen
design 에 명시. p9-fb-01/02/03 (ingest progress callback / CLI display
/ TUI background) 의 spec PR — impl PR 들이 이어진다.

변경:
- docs/wire-schema/v1/ingest_progress.schema.json (신규):
  line-delimited streaming event schema. discriminated by `kind`
  (scan_started → scan_completed → asset_started → asset_finished* →
  embed_batch_* → completed | aborted). 마지막 줄은 기존
  ingest_report.v1 그대로 (외부 wrapper backward-compat).
- 2026-04-27-kebab-final-form-design.md §2.4a (신규):
  IngestProgressEvent 절. 이벤트 ordering / aborted 의 idempotency /
  CLI 의 stderr vs stdout 분리 / TUI · desktop 의 in-memory 소비.
- 2026-04-27-kebab-final-form-design.md §10:
  long-running 작업 (ingest, future eval run, RAG streaming, embed
  batch) 의 두 invariant — progress 의 단일 source / cooperative
  cancel + step boundary. trait (§7.2) 시그니처는 무영향 — facade
  hidden parameter 로 추가.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 19:14:37 +00:00
233b708624 feat(cli/wire): add reset_report.v1 schema + wire_reset helper
JSON Schema 7 frozen surface for `kebab reset --json`. Mirrors the
ResetReport struct from kebab-app. Test asserts schema_version tag,
scope serialization (snake_case enum), removed_paths array, and
embedding_rows_truncated u64.

p9-fb-06 task 3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 18:28:24 +00:00
a166b7051c p0-1: wire-schema stubs, doc/spec stubs, V001 migration, fixtures
- docs/wire-schema/v1/ ships 7 schema stubs (citation, search_hit,
  answer, ingest_report, doc_summary, chunk_inspection, doctor) that
  pin schema_version + required fields per design §2. Full property
  validation lands in later phases.
- docs/spec/ ships 7 markdown stubs each linking to the canonical
  frozen design (domain-model, ids, canonical-document, chunk-policy,
  citation-policy, module-boundaries, ai-generation-guidelines).
- migrations/V001__init.sql contains only schema_meta + migrations
  tables per design §5.1; data tables ship in P1-6/P2-1/P3-3.
- fixtures/ has the 11 subdirectories every downstream task references
  (markdown, source-fs, search/{lexical,hybrid}, embed, vector, rag,
  eval, image, pdf, audio). Empty subdirs use .gitkeep so they track.
  fixtures/markdown/ ships the 3 phase-0 fixtures: simple-note.md,
  nested-headings.md, code-and-table.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 05:17:32 +00:00