Commit Graph

27 Commits

Author SHA1 Message Date
7a6a24ad10 feat(p10-1a-2): add MediaType::Code(lang) variant
TDD: red → green cycle confirmed. New `Code(String)` variant serializes
as `{"code":"rust"}` via serde `rename_all = "lowercase"`. All exhaustive
`match` sites updated (`media_label`, `ingest_one_asset` catch-all →
explicit or-pattern). Design §3.5 enum listing synced. Also fix
`/target` symlink gitignore pattern so integration-test binary lookup
via workspace-relative path works with CARGO_TARGET_DIR redirect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:14:45 +00:00
9f3edb7e24 feat(p10-1a-2): add internal SourceSpan::Code variant + design §3.4 sync
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 15:52:01 +00:00
th-kim0823
bf4ebf8d2a feat(p10-1a-1): add Metadata.repo / git_branch / git_commit / code_lang
Four optional, serde-skipped-when-None fields added to `Metadata` for
code ingest context. All 11 downstream construction sites patched with
`repo: None, git_branch: None, git_commit: None, code_lang: None`.
Full workspace check (`--tests`) and per-crate test suite pass clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 15:44:18 +09:00
th-kim0823
351c7a0826 feat(p10-1a-1): add IngestReport skip counters + SkipExamples
Adds five new u32 counters (skipped_gitignore, skipped_kebabignore,
skipped_builtin_blacklist, skipped_generated, skipped_size_exceeded)
and a SkipExamples struct (≤5 sample paths per category) to
IngestReport. All new fields are #[serde(default)] for backward-compat
deserialization. Downstream literal construction sites patched with
zeros/empty; snapshot re-baked.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 15:28:19 +09:00
th-kim0823
fa4eeb5a87 feat(p10-1a-1): add SearchHit.repo / code_lang + SearchFilters.repo / code_lang
Wire two new optional fields onto SearchHit (skip_serializing_if = None)
and two Vec<String> filter fields onto SearchFilters (serde default).
Add RetrievalDetail::Default impl (manual, uses SearchMode::Hybrid as
sentinel). Patch all downstream SearchHit / SearchFilters literal
constructors with repo: None / code_lang: None / vec![] as appropriate.
Also covers Citation::Code arm in kebab-eval metrics match.
2026-05-15 15:04:23 +09:00
th-kim0823
3b1e878aed feat(p10-1a-1): add Citation::Code variant
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 14:39:18 +09:00
th-kim0823
892175d009 feat(core): BulkSearchItem / Summary / Response types (fb-42) 2026-05-10 20:12:31 +09:00
th-kim0823
3c605b1a5d feat(core): ScoreKind enum + SearchHit.score_kind (fb-38) 2026-05-10 17:49:02 +09:00
th-kim0823
6a33d08aea fix(fb-37): address PR #129 round 1 review
- doc TraceFusionInput.fusion_score semantics (single-mode vs hybrid)
- comment why total_ms vs stage sum can drift (millis truncation)
- TODO marker on TUI trace popup filter passthrough

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:26:34 +09:00
th-kim0823
69c6e23432 feat(store): breakdowns + index_bytes helpers (fb-37)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 12:24:43 +09:00
th-kim0823
1e943f21dc feat(core): SearchTrace + IndexBytes types + SearchOpts.trace (fb-37)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 12:17:04 +09:00
th-kim0823
d3f38c76e9 feat(core): SearchFilters gains media / ingested_after / doc_id (fb-36)
3 additive optional fields. #[serde(default)] preserves
backwards compat for older JSON without the new keys.
MEDIA_KINDS const exposes canonical "markdown"/"pdf"/"image"/
"audio"/"other" labels for downstream alias normalization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 03:36:45 +09:00
th-kim0823
9653592c16 feat(core): FetchQuery / FetchOpts / FetchResult / FetchKind (fb-35)
Domain types for `kebab fetch` 3 modes (chunk / doc / span). All
types Serialize so wire layers hand them through serde_json
directly. FetchKind is snake_case-renamed to match the wire
discriminator literal in fetch_result.v1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 23:35:21 +09:00
th-kim0823
e00418537f feat(core): SearchOpts domain type for budget controls (fb-34)
3 optional knobs (max_tokens, snippet_chars, cursor); Default = all
None = no enforcement (backwards-compat existing search behavior).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 17:46:40 +09:00
th-kim0823
307fd8d527 feat(rag): pipeline emits StreamEvent + cancel on SendError (fb-33)
RetrievalDone after retrieve+stale-stamp, Token per LM chunk
(SendError → break, FinishReason::Cancelled, RefusalReason::
LlmStreamAborted), Final on success. answers row still persists
on cancel for audit. Adds FinishReason::Cancelled, re-exports
StreamEvent from kebab_rag, migrates two pre-fb-33 sink tests
in tests/pipeline.rs to the new StreamEvent type (the
"dropped receiver does not abort" test inverts to record cancel).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 14:49:55 +09:00
th-kim0823
3a7a28e682 feat(core): AnswerCitation gains indexed_at + stale (fb-32)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:56:36 +09:00
th-kim0823
8b0f64db6b feat(core): SearchHit gains indexed_at + stale (fb-32)
Domain field additions for p9-fb-32. Wire serialization is
automatic via serde rfc3339. Other crates fail to compile until
they populate the new fields — fixed in subsequent tasks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:52:46 +09:00
693f5582f0 feat(kebab-core, kebab-app): p9-fb-25 task 4 — IngestReport.skipped_by_extension + wire schema additive
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:06:34 +00:00
366e89e5e2 feat(kebab-store-sqlite): p9-fb-23 task 4 — get_asset_by_workspace_path
Add `DocumentStore::get_asset_by_workspace_path` trait method to
`kebab-core` and implement it on `SqliteStore` via a private
`asset_from_row` helper. Used by the incremental-ingest skip path to
compare a freshly-computed blake3 checksum against the persisted row
without a full round-trip through `put_asset_with_bytes`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 17:58:23 +00:00
f867b36afb feat(kebab-core): p9-fb-23 task 2 — CanonicalDocument gains last_chunker_version + last_embedding_version
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 17:50:25 +00:00
aa2a6ea7fc feat(kebab-core): p9-fb-23 task 1 — IngestItemKind::Unchanged + IngestReport.unchanged
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 17:43:52 +00:00
c97e8e00ef feat(kebab-core + kebab-store-sqlite): p9-fb-17 chat session storage (V005)
도그푸딩 item 13/14 (multi-turn 영속화) — TUI Ask 의 "이전 대화
이어가기" + 향후 CLI `--session foo` (p9-fb-18) backing store. session
header + per-turn 두 테이블, ON DELETE CASCADE 로 reset --data-only 가
한꺼번에 wipe.

## 핵심 변경

- **SQLite V005 migration** `chat_sessions` (session_id PK + created_at
  + updated_at + title + config_snapshot_json) + `chat_turns` (turn_id
  PK + session_id FK ON DELETE CASCADE + turn_index + question +
  answer + citations_json + created_at + UNIQUE(session_id, turn_index))
  + `idx_chat_turns_session(session_id, turn_index)`. 모두 `STRICT`.
- **`kebab_core::ChatSessionRepo`** trait (6 method): create_session /
  get_session / list_sessions(limit, ORDER BY updated_at DESC) /
  delete_session / append_turn / list_turns(ORDER BY turn_index ASC)
- **`kebab_core::{ChatSessionRow, ChatTurnRow}`** structs — Serialize
  + Deserialize 둘 다 (CLI / wire 출력 호환)
- **`kebab-store-sqlite::SqliteStore`** impl 신규 모듈 `chat_sessions.rs`.
  `append_turn` 이 insert + parent updated_at bump 같은 connection
  에서 처리.
- **frozen design §5** 에 §5.7a chat_sessions / chat_turns 절 신설
  (full schema + trait 메서드 6 개 명시).

## HOTFIXES (V004 → V005)

spec p9-fb-17 의 `V004__chat_sessions.sql` 가 p9-fb-19 의
`V004__kv.sql` (이미 머지) 와 refinery migration number 충돌. 무중단
정정: `V005__chat_sessions.sql` 로 시프트. schema / 동작 동일, 파일명
만 이동. HOTFIXES entry 추가.

## 테스트

- 9 신규 integration unit (create/get roundtrip, missing→None, PK
  collision error, append+list ordered, dup turn_index error,
  append bumps updated_at, delete CASCADE turns, list_sessions
  ORDER BY updated_at DESC, list_sessions LIMIT)
- workspace 전체 `cargo test --workspace --no-fail-fast -j 1` exit 0
- `cargo clippy --workspace --all-targets -- -D warnings` clean

## 문서

- frozen design §5.7a 신설
- HANDOFF: 2026-05-03 entry
- HOTFIXES: V004 → V005 rename rationale
- spec status planned → in_progress

## Out of scope

- session 검색 / 필터 UI (p9-fb-18 의 `kebab ask --session list`
  같은 admin command 가 후속)
- 다른 store backend (postgres 등) — trait 만 정의, impl 은 SQLite

unblocks p9-fb-18 (CLI session/repl).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:37:53 +00:00
2c058ab175 feat(rag): multi-turn ask — Turn struct + ask_with_history + token budget (p9-fb-15)
Spec PR #59 의 §3.8 multi-turn behaviour 구현. RAG facade 가 prior
turns 받아 prompt 에 prepend, retrieval query expansion 적용,
Answer 에 conversation_id / turn_index 채움.

신규 (kebab-core):
- Answer 에 conversation_id (Option<String>) / turn_index (Option<u32>)
  field 추가. serde skip_serializing_if 로 single-shot 의 wire
  output 변경 0 (기존 외부 wrapper 영향 없음).
- Turn struct (question + answer + citations + created_at).
- RefusalReason::LlmStreamAborted variant.

신규 (kebab-rag):
- AskOpts 에 history (Vec<Turn>) / conversation_id / turn_index 3 field.
- AskOpts::single_shot(mode) helper.
- RagPipeline::ask_with_history(query, history, conversation_id,
  turn_index, opts) — combined opts 로 ask 호출.
- expand_query_with_history: history.last() 의 answer 첫 200 자
  concat 해 SearchQuery.text 확장 (spec §3.8 의 \"cheap concat\";
  LLM-based standalone-question rewriting 은 P+).
- serialize_history + remaining_history_budget_chars: spec 의 priority
  enforcement — system+question 필수, retrieved chunks 가 차지한
  뒤 남은 char budget 안에서 newest 우선, oldest drop.
- ask 본문: history 가 비어있지 않으면 [이전 대화] 블록을 user
  prompt 위에 prepend. Answer 생성 site 3 곳 (정상 / NoChunks /
  ScoreGate refuse) 모두 conversation_id / turn_index 채움.

신규 (kebab-store-sqlite):
- refusal_reason_label 가 LlmStreamAborted → 'llm_stream_aborted'.

기존 caller 변경 (single-shot 동작 동일):
- kebab-cli main.rs Cmd::Ask: AskOpts 에 history=Vec::new(),
  conversation_id=None, turn_index=None 명시 (CLI multi-turn 은
  p9-fb-18 의 --session/--repl 가 채움).
- kebab-tui src/ask.rs spawn site 동일 (multi-turn UI 는 p9-fb-16).
- kebab-eval runner.rs golden eval 동일 (single-shot per query).
- kebab-app tests/ask_smoke.rs / kebab-tui tests/ask.rs / kebab-rag
  tests/pipeline.rs / kebab-eval metrics.rs Answer literal 갱신.

Test:
- 9 신규 lib unit (expand_query 4 / serialize_history 3 / remaining_budget 2).
- 기존 12 PASS 회귀 0.

Plan 갱신:
- p9-fb-15 status planned → in_progress. 머지 후 한 줄 commit
  으로 completed flip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:09:46 +00:00
0c8821f857 fix(kebab-store-vector): close P7-3 vector orphan caveat — delete_by_chunk_ids
P7-3 의 storage UNIQUE bug fix 가 SQLite 측 (documents → blocks /
chunks / embedding_records) 만 sweep 했음. LanceDB 의 vector 는 별도
store 라 옛 chunk_id 를 가진 row 가 디스크에 잔존. 검색에는 영향 없지만
디스크는 무한 누적. HOTFIXES `2026-05-02 P7-3` caveat 의 "P+ task" 약속을
같은 후속 PR 안에서 닫음.

변경:
- `VectorStore::delete_by_chunk_ids(&[ChunkId])` trait method 추가 (default
  no-op 제공 — 테스트 fake / 기존 impl 이 그대로 컴파일).
- `LanceVectorStore::delete_by_chunk_ids` 가 connection 의 모든
  `chunk_embeddings_*` 테이블을 순회 + `Table::delete("chunk_id IN (...)")`
  를 batch=200 단위로 실행. 다중 모델 워크스페이스 (마이그레이션 중간 등)
  에서도 안전.
- `SqliteStore::stale_chunk_ids_at(workspace_path, new_asset_id)` 가
  read-only SELECT 로 옛 chunk_id 들 반환. CASCADE 가 흐르기 *전* 에
  caller 가 호출.
- `kebab-app::purge_vector_orphans_for_workspace_path` 가 위 두 단계를
  orchestrate. 세 ingest path (markdown / image / pdf) 의
  `put_asset_with_bytes` 호출 직전에 한 줄로 호출.

Smoke 검증 (release binary, fastembed enabled):
- whitepaper.pdf 첫 ingest → chunk_ids = {f616…, 4e0f…}, vector store 에
  그 두 ID 의 row 존재.
- byte 변경 후 re-ingest → 새 doc_id (3741…) + 새 chunk_ids
  (ed0c…, e13c…). vector search "REWRITTEN chapter two" → 새 chunk_ids 만
  hit. 옛 query "Edited page two body" 시도해도 옛 chunk_ids 는 vector
  store 에 더 이상 없음 (의미적으로 가장 가까운 새 chunks 가 hit).

HOTFIXES `2026-05-02 P7-3` 의 \"vector store cleanup\" 항목이 \"deferred\" →
\"closed by follow-up PR\" 로 갱신. SMOKE.md 의 알려진 동작 (\"옛 vector
잔존\") 도 \"두 store 정합\" 으로 갱신.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 12:32:29 +00:00
cd2213e48d feat(kebab-parse-image): P6-3 caption adapter — vision LM via trait
- 신규 모듈 `crates/kebab-parse-image/src/caption.rs` 추가:
  • `caption_image(llm, bytes, lang_hint, cfg)` — `&dyn LanguageModel`
    위에서 동작. 비전 LM (예: gemma4:e4b) 이 한 문장 객관 설명
    출력. temperature=0 / seed=0 결정성.
  • `apply_caption(llm, bytes, block, lang_hint, cfg, events)` —
    `block.caption = Some(...)` 으로 채우고 ProvenanceKind::CaptionApplied
    이벤트 1건 추가. `image.caption.enabled = false` 면 클린 no-op
    (Ok(())). LM 실패 시 block.caption None 그대로 + events 미기록.
  • 다운스케일 long-edge `[128, 1536]` 클램프. PNG passthrough hot
    path 보존, 그 외는 단일 디코드 + PNG 재인코딩.
  • 한국어 / 영어 프롬프트 분기 (lang_hint=\"ko\"/\"kor\" → 한국어).
  • `ModelCaption.model_version = \"<provider>/<prompt_template_version>\"`
    (예: \"ollama/caption-v1\") — prompt 또는 모델 회귀 감사 가능.

## kebab-core / kebab-llm-local 변경

- `kebab_core::GenerateRequest` 에 `images: Vec<String>` 필드 추가.
  `#[serde(default)]` 으로 기존 wire 페이로드 / snapshot 호환.
- `kebab-llm-local::OllamaLanguageModel` 가 req.images 를 Ollama
  `images: [base64, ...]` 와이어 필드로 라우팅.
  `#[serde(skip_serializing_if = is_empty)]` 로 비어 있을 때 wire
  shape 가 pre-P6-3 와 byte-identical.

## kebab-config

- 신규 `ImageCfg.caption: CaptionCfg`:
  - `enabled: bool` (default false)
  - `max_pixels: u32` (default 768, 클램프 [128, 1536])
  - `prompt_template_version: String` (default \"caption-v1\")
- `KEBAB_IMAGE_CAPTION_{ENABLED,MAX_PIXELS,PROMPT_TEMPLATE_VERSION}`
  3종 환경변수 추가.

## Spec deviations

`tasks/HOTFIXES.md` 2026-05-02 항목 추가:
- Symptom 1: spec p6-3 시그니처가 `&dyn LanguageModel` 인데 frozen
  trait + GenerateRequest 가 vision 미지원. → trait 확장.
- Symptom 2: spec 의 cargo feature `caption` (default OFF at compile
  time) → runtime gate 1개로 통합. base64/image/kebab-llm 외 추가
  deps 없어 cargo feature 의 binary 절감 가치 미미.

p4-1 / p4-2 / p6-3 spec 의 amends 명시.

## 테스트

`cargo test -p kebab-parse-image --test caption` — 9건 + 1 ignored:
- feature gate (disabled → no-op / Err on direct call)
- happy path (block.caption Some + Provenance CaptionApplied)
- 빈 토큰 stream → empty text + caption.is_some()
- CapturingMock 으로 req.images 라우팅 검증 (base64 1개, decode 가능)
- 한국어 / 영어 프롬프트 분기 (CapturingMock 의 system 캡처)
- LM Err → block.caption None 유지 + events 미기록
- 결정성 (동일 mock 입력 → 동일 caption)
- max_pixels 클램프 (99999 → 1536, 4000×3000 PNG 다운스케일 검증)
- opt-in 통합 (실 192.168.0.47 Ollama / gemma4:e4b → \"The image is
  a solid red color.\" 검증 완료, 4.3초)

`cargo test --workspace --no-fail-fast -j 1` 전체 pass.
`cargo clippy --workspace --all-targets -- -D warnings` pass.

## 의존성 경계

- 추가 deps: `kebab-llm` (trait 만), `base64` (이미 P6-2 에서 추가).
- dev-deps: `kebab-llm/mock` 으로 `MockLanguageModel`,
  `kebab-llm-local` (통합 테스트 전용 — 런타임 deps 에는 없음).
- forbidden 침범 없음: `kebab-source-fs / parse-md / normalize /
  chunk / store-* / embed* / search / rag / UI` 미참조.

contract: docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
sections: §3.4 ImageRefBlock.caption, §3.7a ModelCaption, §9.1
caption (model-generated, low trust).
2026-05-02 06:05:39 +00:00
f1a448d6dc refactor(rename): kb → kebab — binary, env vars, XDG paths, file renames
두 번째 commit. 사용자 facing surface (CLI binary, env vars, XDG paths)
+ 코드 안 single-letter token (`KB_`, `kb.sqlite`, `/kb/`, tracing
target) 일괄 rename. 그리고 3 개 file rename:

- 디자인 doc `2026-04-27-kb-final-form-design.md` →
  `2026-04-27-kebab-final-form-design.md`
- 최초 보고서 `kb_local_rust_report.md` → `kebab_local_rust_report.md`
- workspace ignore `.kbignore` → `.kebabignore`

## 변경

- `crates/kebab-cli/Cargo.toml`: `[[bin]] name = "kb"` → `"kebab"`.
- `crates/kebab-cli/src/main.rs`: `#[command(name = "kb", …)]` →
  `name = "kebab"`.
- 모든 `KB_*` env var (코드 + doc + 테스트) → `KEBAB_*`. apply_env
  prefix 매칭 + 30+ 개 setting 키 모두.
- XDG paths: `~/.config/kb` / `~/.local/share/kb` / `~/.cache/kb` /
  `~/.local/state/kb` → `~/.config/kebab` 등. config defaults +
  expand_path tests + paths.rs 의 hardcode 모두.
- SQLite filename: `kb.sqlite` → `kebab.sqlite` (`SQLITE_FILE` const
  + 테스트 hardcode 모두).
- tracing target: `target: "kb-*"` → `"kebab-*"` (10+ 곳).
- snapshot fixture: `.kbignore` → `.kebabignore` (`fixtures/source-fs/
  tree-1.snapshot.json` 갱신).

## 검증

- `cargo test --workspace -j 1` clean (linker OOM 회피 위해 직렬).
- `cargo clippy --workspace --all-targets -- -D warnings` clean.

다음 commit 에서 docs sweep.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 04:01:35 +00:00
911fb49550 refactor(rename): kb crates → kebab — Cargo packages, folders, Rust modules
프로젝트 이름 `kb` → `kebab` rename 의 첫 단계.

- workspace `Cargo.toml`: members `crates/kb-*` → `crates/kebab-*`,
  repository URL `altair823/kb` → `altair823/kebab`.
- 18 crate 폴더 rename via `git mv` (history 보존).
- 각 crate `Cargo.toml`: `name = "kb-*"` → `"kebab-*"`, path deps
  `../kb-*` → `../kebab-*`.
- 모든 `.rs`: `kb_<id>` snake-case 모듈 path 18 개 (`kb_core`,
  `kb_config`, `kb_app`, `kb_cli`, `kb_eval`, `kb_search`, `kb_chunk`,
  `kb_normalize`, `kb_source_fs`, `kb_parse_md`, `kb_parse_types`,
  `kb_store_sqlite`, `kb_store_vector`, `kb_embed`, `kb_embed_local`,
  `kb_llm`, `kb_llm_local`, `kb_rag`) → `kebab_<id>` 일괄 sed (단어
  경계 \\b 사용해 영어 문장 안의 "kb" 약어 미오염).

CLI binary 이름 (`[[bin]] name = "kb"`), 환경변수 `KB_*`, XDG paths,
tracing target, 그리고 docs sweep 은 다음 commit 에서.

## 검증

- `cargo check --workspace` clean — 모든 crate 빌드 통과 후 commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:28:08 +00:00