Merge pull request 'fix(kebab-config): p9-fb-25 — workspace.include 제거 + 지원 형식 가시성' (#101) from fix/p9-fb-25-config-include-removal into main
Reviewed-on: #101
This commit was merged in pull request #101.
This commit is contained in:
@@ -60,6 +60,7 @@ P0~P5 직렬. P6~P9 P5 이후 병렬 가능.
|
||||
- **2026-05-03 P9 도그푸딩 후속 (p9-fb-12 follow-up)** — heuristic 제거 (partial PR 의 deferred 부분 finalize). `search::is_typing_mod` (CTRL/ALT chord filter) 함수 삭제 + `ask::handle_key_ask` 의 input-empty heuristic 삭제. 새 dispatch: `search::handle_key_search` 의 `i` (chunk inspect) / `g` (editor jump) pre-pass 가 `state.mode == Mode::Normal` 일 때만 fire (Insert 에서는 typed char). main match 의 `j`/`k`/Char(c) 가 `state.mode` 로 분기 (Normal → 선택 이동, Insert → input.push). `ask::handle_key_ask` 의 `e`/`j`/`k` 도 동일 패턴 — Normal 에서 toggle/scroll, Insert 에서 input typing. 테스트 fixture (`tests/search.rs::fresh_app`, `tests/ask.rs::fresh_app`) 가 `app.mode = Mode::auto_for(focus)` 로 run-loop 동작 mirror. 기존 nav 테스트 (j_k_move, g_key_enqueues, e_toggles) 는 explicit `app.mode = Mode::Normal` 추가, 신규 4 테스트 (j_in_insert_types / arbitrary_char_in_normal_noop / e_types_in_insert / jk_scroll-in-normal-type-in-insert) 가 mode-authoritative 동작 pin. spec status `in_progress` → `completed`. spec: `tasks/p9/p9-fb-12-tui-mode-machine.md`.
|
||||
- **2026-05-03 P9 도그푸딩 후속 (p9-fb-10 partial)** — TUI CJK rendering helpers. `kebab-tui::input::{display_width, truncate_to_display_width}` 신규 — `unicode-width` 위에서 column-단위 width 계산 (ASCII=1, Hangul/CJK/fullwidth=2, combining=0) + char-boundary 안전 truncate (wide char 를 split 없이 keep-or-omit, ellipsis 1 col). library.rs 의 중복 `truncate_to_display_width` private fn 제거 — 단일 source. 9 unit tests (ASCII / Hangul / Japanese / mixed / truncate fits·overflow·zero-cols·wide-char-boundary / `String::pop` char-aware sanity) + 1 integration render test (Korean + Japanese fixture, TestBackend 80×20, 한글/일본어 글자가 frame 에 살아남음 확인). spec 의 `InputBuffer` struct (cursor 가 column 단위 wide-char width 추적) 도입은 follow-up — Ask/Search/Editor pane 의 String + cursor 일괄 마이그레이션이 회귀 표면이 커서 helper 만 먼저 머지. backspace 는 모든 pane 이 이미 `String::pop()` 사용 (char-aware) → byte-boundary 안전성 helper 없이도 확보. crossterm 0.28 이 native IME composing 미노출 — preedit handling out of scope. spec status `planned` → `in_progress`. spec: `tasks/p9/p9-fb-10-tui-cjk-input.md`.
|
||||
- **2026-05-04 P9 post-도그푸딩 (p9-fb-23)** — Incremental ingest. 사용자 도그푸딩 피드백: 변하지 않은 문서는 다시 ingest 하지 않기. blake3 checksum + parser_version + chunker_version + embedding_version 4개 input 이 모두 일치할 때 parse/chunk/embed/vector upsert 모두 회피. SQLite V006 마이그레이션 — `documents` 에 `last_chunker_version` + `last_embedding_version` 컬럼 추가. 신규 `IngestItemKind::Unchanged` variant + `IngestReport.unchanged` + `AggregateCounts.unchanged` (wire schema additive). `IngestOpts { progress, cancel, force_reingest }` struct 도입 — `AskOpts` 패턴. `--force-reingest` CLI flag 로 skip 우회. 비용 dominator (fastembed) 가 변경된 / 새 doc 에만 발생. spec: `tasks/p9/p9-fb-23-incremental-ingest.md`. HOTFIXES `2026-05-04 — p9-fb-23` 항목이 version cascade 명시 동작의 source of truth.
|
||||
- **2026-05-05 P9 post-도그푸딩 (p9-fb-25)** — Config 의 `workspace.include` 필드 제거 + 지원 형식 가시성. 사용자 도그푸딩 피드백: include + exclude 동시 존재가 case 4 (둘 다 매치 안 함) 의미 모호 + 어차피 처리 가능 형식 (md / png / jpg / pdf) 이 정해져 있으니 명시 필요. `WorkspaceCfg.include` 제거 (옛 config 의 `include = [...]` 은 silently 무시 + 단발 deprecation warning). `IngestItem.warnings` 가 Skipped 시 사유 (`"unsupported media type: .docx"` 등) 채움. `IngestReport.skipped_by_extension: BTreeMap<String, u32>` 신규 (additive wire — release 트리거 안 됨). CLI / TUI summary 에 breakdown 표시 (`"5 skipped: 3 docx, 1 txt, 1 epub"`). README + `kebab init` 헤더 주석에 지원 형식 명시. spec: `tasks/p9/p9-fb-25-config-include-removal.md`. HOTFIXES `2026-05-05 — p9-fb-25` 가 source of truth.
|
||||
- **2026-05-04 P9 post-도그푸딩 (p9-fb-24)** — TUI status/key bar + Library 컬럼 헤더 + Ask/Inspect PgUp/PgDn. 사용자 도그푸딩 3 건 (Library 컬럼 의미 부재, 페이지 스크롤 키 부재, 상태바 + 버전 정보 항상 노출 요청) 을 단일 PR 로 통합. bottom 영역을 status bar (1 row, version + pane + docs + dynamic state) + key hint bar (1 row, 기존 `footer_hints` 그대로) 두 줄로 분할; 기존 ingest progress dedicated row 는 status bar 의 dynamic slot 에 흡수 (priority cascade: streaming → searching → indexing → idle). Library `List` 위에 `format_doc_header` 행 + Layout 분할로 헤더 표시 (TITLE / TAGS / UPDATED / CHUNKS, display-width 정렬). `kebab-tui::pager::PAGE_STEP = 10` 신규 — Ask 의 PgUp/PgDn 추가 + Inspect 의 기존 +/-10 hardcode 가 같은 상수 참조로 통일. Ask 의 page-scroll 은 `j`/`k` 와 동일하게 `follow_tail = false` 로 freeze. spec: `tasks/p9/p9-fb-24-tui-affordances.md`. HOTFIXES `2026-05-04 — p9-fb-24` 항목이 footer 단행 row (p9-fb-13) + ingest dedicated row (p9-fb-03) 와의 layout 충돌의 source of truth.
|
||||
- **2026-05-04 P9 post-도그푸딩 (p9-fb-22)** — TUI 입력 cursor mid-string 편집 + Ask follow-tail auto-scroll. Gitea #94 (입력 후 커서 이동 안 됨) + #95 (새 응답 자동 스크롤 안 됨) 두 건. `InputBuffer` 의 cursor 모델을 byte-position 기반으로 재구성 — cursor 가 끝일 때 기존 append 동작과 backwards-compatible, mid-string 일 때는 `←/→/Home/End/Delete` 로 편집. `AskState` 에 `follow_tail: bool` (default true). `Paragraph::line_count(width)` (ratatui `unstable-rendered-line-info` feature 활성화) 로 매 프레임 wrapped row 수 계산해 follow-tail 시 scroll 을 bottom 에 pin. `j`/`k` 가 follow-tail 끄고 `Shift-G` 가 다시 켬. 12 신규 InputBuffer unit + 6 신규 Ask integration. spec: `tasks/p9/p9-fb-22-tui-cursor-and-autoscroll.md`. HOTFIXES 항목 `2026-05-04` 가 live cursor 모델 source of truth.
|
||||
- **2026-05-03 P9 post-도그푸딩 (p9-fb-21)** — `i` 가 universal Normal→Insert toggle (모든 pane). 이전 mode_intercept 는 Library/Inspect/Jobs 만 `i` intercept 였고 Search/Ask 는 fall-through (자동 INSERT 가정). 사용자가 Esc 로 NORMAL 로 빠진 후 Insert 복귀 키 없어 dead-end → 도그푸딩에서 보고됨. mode_intercept 의 `(Char('i'), Normal, _)` arm 이 pane 무관 모두 INSERT flip. Search 의 chunk inspect 키 `i`→`o` rebind (vim "open") 으로 충돌 해소. footer hint 모든 (pane, mode, filter) 조합 첫 fragment = `F1 도움말` (cheatsheet binding discoverability). Search/Ask Normal hint 에 `i 입력모드` fragment 추가. cheatsheet popup Global/Search/Ask section 갱신. 6 신규 unit + 3 기존 갱신. spec: `tasks/p9/p9-fb-21-tui-insert-key-discoverability.md` (status `completed` 직접). HOTFIXES 항목이 Search `i`→`o` rebind 의 source of truth.
|
||||
|
||||
@@ -42,7 +42,7 @@ cargo install --git https://gitea.altair823.xyz/altair823-org/kebab.git --bin ke
|
||||
# 첫 실행 — XDG 경로에 데이터 디렉토리 + config.toml 생성
|
||||
kebab init
|
||||
|
||||
# config 손보고 — `[workspace] include` 에 *.md / *.png / *.pdf 등 추가, 모델 endpoint 등
|
||||
# config 손보고 — workspace.root, 모델 endpoint 등 설정 (지원 형식은 md / png / jpg / pdf 로 고정)
|
||||
${EDITOR:-vi} ~/.config/kebab/config.toml
|
||||
|
||||
# 색인 (Markdown / 이미지 / PDF 모두 한 번에)
|
||||
@@ -70,7 +70,7 @@ kebab doctor
|
||||
| 명령 | 동작 |
|
||||
|------|------|
|
||||
| `kebab init` | XDG 경로에 데이터 디렉토리 + config.toml 생성 |
|
||||
| `kebab ingest [<path>]` | Markdown / 이미지 / PDF 색인 (idempotent). TTY 에서는 stderr 진행 바, non-TTY (CI / pipe) 는 stderr 한 줄씩, `--json` 은 stdout 에 `ingest_progress.v1` 라인 streaming 후 마지막에 `ingest_report.v1`. Ctrl-C 한 번이면 현재 asset 마무리 후 abort (부분 commit 보존, idempotent re-run), 두 번째 Ctrl-C 는 hard exit. Markdown title 이 frontmatter 에 없어도 첫 H1 → H2 → 첫 paragraph 80 자 → 파일명 순으로 자동 채움 (parser_version `md-frontmatter-v2`) — 기존 색인된 doc 도 다음 ingest 에서 새 title 로 갱신. **Incremental** (p9-fb-23): 두 번째 이후의 ingest 는 변하지 않은 doc (blake3 + parser/chunker/embedder version 모두 동일) 의 parse/chunk/embed/vector upsert 를 자동 스킵. final summary 에 `N unchanged` 카운트 표시. `--force-reingest` 로 skip 무시 강제 재처리. |
|
||||
| `kebab ingest [<path>]` | Markdown / 이미지 / PDF 색인 (idempotent). TTY 에서는 stderr 진행 바, non-TTY (CI / pipe) 는 stderr 한 줄씩, `--json` 은 stdout 에 `ingest_progress.v1` 라인 streaming 후 마지막에 `ingest_report.v1`. Ctrl-C 한 번이면 현재 asset 마무리 후 abort (부분 commit 보존, idempotent re-run), 두 번째 Ctrl-C 는 hard exit. Markdown title 이 frontmatter 에 없어도 첫 H1 → H2 → 첫 paragraph 80 자 → 파일명 순으로 자동 채움 (parser_version `md-frontmatter-v2`) — 기존 색인된 doc 도 다음 ingest 에서 새 title 로 갱신. **Incremental** (p9-fb-23): 두 번째 이후의 ingest 는 변하지 않은 doc (blake3 + parser/chunker/embedder version 모두 동일) 의 parse/chunk/embed/vector upsert 를 자동 스킵. final summary 에 `N unchanged` 카운트 표시. `--force-reingest` 로 skip 무시 강제 재처리. **지원 형식** (extractor 자동 결정 — config 에 명시 불가): Markdown (`.md`), 이미지 (`.png` / `.jpg` / `.jpeg`, OCR + caption), PDF (`.pdf`). 다른 확장자는 자동 skip — `IngestItem.warnings` 에 사유 (`"unsupported media type: .docx"` 등), `IngestReport.skipped_by_extension` 에 카운트 분류, CLI / TUI summary 에 breakdown 표시. |
|
||||
| `kebab search --mode {lexical,vector,hybrid} "<query>" [--no-cache]` | 검색. hybrid는 RRF fusion, citation 포함. 같은 process 안에서 동일 query (NFKC + trim + lowercase 정규화) 반복 시 in-process LRU 캐시 hit (capacity = `[search] cache_capacity`, default 256). `--no-cache` 로 강제 bypass — 디버깅용. ingest commit 발생 시 `kv['corpus_revision']` bump 으로 모든 entry 자동 stale |
|
||||
| `kebab list docs` | 색인된 문서 목록 |
|
||||
| `kebab inspect doc <id>` / `kebab inspect chunk <id>` | raw record 보기 |
|
||||
@@ -145,7 +145,7 @@ flowchart TB
|
||||
|
||||
## Configuration
|
||||
|
||||
- `~/.config/kebab/config.toml` — `kebab init` 가 XDG 경로에 생성. `[workspace] include`, `[storage]`, `[chunking]`, `[models.embedding]`, `[models.llm]`, `[image.ocr]`, `[image.caption]`, `[search]`, `[rag]`, `[ui]` 절. `[ui] theme = "dark" | "light"` 로 TUI 팔레트 선택 (default `"dark"`, 알 수 없는 값은 dark fallback).
|
||||
- `~/.config/kebab/config.toml` — `kebab init` 가 XDG 경로에 생성. `[workspace]` (root, exclude — include 필드는 제거됨, 지원 형식은 자동 결정), `[storage]`, `[chunking]`, `[models.embedding]`, `[models.llm]`, `[image.ocr]`, `[image.caption]`, `[search]`, `[rag]`, `[ui]` 절. `[ui] theme = "dark" | "light"` 로 TUI 팔레트 선택 (default `"dark"`, 알 수 없는 값은 dark fallback). 옛 config 의 `workspace.include = [...]` 은 silently 무시 + 단발 deprecation warning (p9-fb-25).
|
||||
- `--config <path>` flag — 임시 워크스페이스 / 격리 테스트 시 사용. CLI / TUI 모두 honor.
|
||||
- `KEBAB_*` env — 일부 키 override (`KEBAB_RAG_SCORE_GATE`, `KEBAB_EVAL_GOLDEN`, `KEBAB_COMMIT_HASH` 등).
|
||||
- XDG layout: `~/.config/kebab/`, `~/.local/share/kebab/`, `~/.cache/kebab/`, `~/.local/state/kebab/`.
|
||||
|
||||
@@ -22,7 +22,7 @@ use kebab_core::IngestItemKind;
|
||||
/// `p9-fb-04`, `Aborted`) events. Mirrors the fields persisted into
|
||||
/// `ingest_runs.progress_json` so external tooling can reconstruct the
|
||||
/// run's outcome from either side.
|
||||
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq, Serialize, Deserialize)]
|
||||
#[derive(Clone, Debug, Default, Eq, PartialEq, Serialize, Deserialize)]
|
||||
pub struct AggregateCounts {
|
||||
pub scanned: u32,
|
||||
pub new: u32,
|
||||
@@ -35,6 +35,8 @@ pub struct AggregateCounts {
|
||||
pub errors: u32,
|
||||
pub chunks_indexed: u32,
|
||||
pub embeddings_indexed: u32,
|
||||
/// p9-fb-25: per-extension skip count. See [`IngestReport::skipped_by_extension`].
|
||||
pub skipped_by_extension: std::collections::BTreeMap<String, u32>,
|
||||
}
|
||||
|
||||
/// One streaming progress event. The CLI's `--json` mode serializes this
|
||||
@@ -98,6 +100,20 @@ pub fn media_label(media: &kebab_core::MediaType) -> &'static str {
|
||||
}
|
||||
}
|
||||
|
||||
/// p9-fb-25: render `": A docx, B txt"` breakdown after the
|
||||
/// `N skipped` count when the map is non-empty. Empty → empty
|
||||
/// string (no extra punctuation). desc sort by count, ties broken
|
||||
/// by key alphabetic.
|
||||
pub fn render_skipped_breakdown(map: &std::collections::BTreeMap<String, u32>) -> String {
|
||||
if map.is_empty() {
|
||||
return String::new();
|
||||
}
|
||||
let mut entries: Vec<_> = map.iter().collect();
|
||||
entries.sort_by(|a, b| b.1.cmp(a.1).then_with(|| a.0.cmp(b.0)));
|
||||
let parts: Vec<String> = entries.iter().map(|(k, v)| format!("{v} {k}")).collect();
|
||||
format!(": {}", parts.join(", "))
|
||||
}
|
||||
|
||||
/// Best-effort send into an optional `mpsc::Sender`. A dropped receiver
|
||||
/// is silently absorbed — the ingest hot path must not stall on a slow
|
||||
/// consumer. Logged at `trace` for diagnostics.
|
||||
@@ -192,4 +208,19 @@ mod tests {
|
||||
other => panic!("unexpected event: {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_skipped_breakdown_desc_sort_with_tiebreak() {
|
||||
use std::collections::BTreeMap;
|
||||
let mut m = BTreeMap::new();
|
||||
assert_eq!(render_skipped_breakdown(&m), "");
|
||||
m.insert("txt".to_string(), 1);
|
||||
m.insert("docx".to_string(), 2);
|
||||
m.insert("epub".to_string(), 1);
|
||||
// 2 docx 먼저 (count desc), 그 다음 1 epub / 1 txt 는 alphabetic.
|
||||
assert_eq!(
|
||||
render_skipped_breakdown(&m),
|
||||
": 2 docx, 1 epub, 1 txt".to_string()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,9 +61,16 @@ pub mod logging;
|
||||
pub mod reset;
|
||||
|
||||
pub use app::App;
|
||||
pub use ingest_progress::{AggregateCounts, IngestEvent};
|
||||
pub use ingest_progress::{AggregateCounts, IngestEvent, render_skipped_breakdown};
|
||||
pub use reset::{ResetReport, ResetScope};
|
||||
|
||||
/// p9-fb-25: sentinel for files without an extension in
|
||||
/// `IngestReport.skipped_by_extension` keys + `IngestItem.warnings`
|
||||
/// `unsupported media type: ...` line. Wire schema description
|
||||
/// references this literal — changing the sentinel is a wire-
|
||||
/// compatibility break.
|
||||
pub const NO_EXT_SENTINEL: &str = "<no-ext>";
|
||||
|
||||
/// Parser-version label persisted in `documents.parser_version` for
|
||||
/// every Markdown file ingested through the `kb-parse-md` pipeline.
|
||||
/// Kept in lock-step with the literal used in the `kb-store-sqlite`
|
||||
@@ -146,6 +153,14 @@ pub fn init_workspace(force: bool) -> anyhow::Result<()> {
|
||||
# — relative paths resolve against the directory of THIS
|
||||
# config file, NOT the user's `cwd` at invocation time.
|
||||
#
|
||||
# 처리 가능한 형식 (extractor 가 자동 결정 — config 에 명시할 수 없음):
|
||||
# • Markdown: .md
|
||||
# • 이미지: .png .jpg .jpeg (OCR + caption)
|
||||
# • PDF: .pdf
|
||||
# 다른 확장자는 ingest 시 자동 skip + warning. 처리 대상 폴더의
|
||||
# 일부만 ingest 하고 싶으면 `kebab ingest <path>` 로 root 명시
|
||||
# 또는 `.kebabignore` 파일 / 본 `workspace.exclude` 로 denylist.
|
||||
#
|
||||
# Override individual keys at runtime with `KEBAB_*` env vars
|
||||
# (e.g. `KEBAB_WORKSPACE_ROOT=/tmp/test kebab ingest`).
|
||||
\n";
|
||||
@@ -375,6 +390,9 @@ pub fn ingest_with_config_opts(
|
||||
// without re-walking the DB.
|
||||
let mut chunks_indexed: u32 = 0;
|
||||
let mut embeddings_indexed: u32 = 0;
|
||||
// p9-fb-25: per-extension skip count, populated in the Skipped arm below.
|
||||
let mut skipped_by_extension: std::collections::BTreeMap<String, u32> =
|
||||
std::collections::BTreeMap::new();
|
||||
let scanned_count: u32 = u32::try_from(assets.len()).unwrap_or(u32::MAX);
|
||||
|
||||
let embed_active = embedder.is_some() && vector_store.is_some();
|
||||
@@ -464,7 +482,9 @@ pub fn ingest_with_config_opts(
|
||||
}
|
||||
}
|
||||
kebab_core::IngestItemKind::Skipped => {
|
||||
skipped_count = skipped_count.saturating_add(1)
|
||||
skipped_count = skipped_count.saturating_add(1);
|
||||
let ext = ext_for_skip_warning(&item.doc_path.0);
|
||||
*skipped_by_extension.entry(ext).or_insert(0) += 1;
|
||||
}
|
||||
kebab_core::IngestItemKind::Unchanged => {
|
||||
unchanged_count = unchanged_count.saturating_add(1)
|
||||
@@ -613,6 +633,7 @@ pub fn ingest_with_config_opts(
|
||||
errors: error_count,
|
||||
chunks_indexed,
|
||||
embeddings_indexed,
|
||||
skipped_by_extension: skipped_by_extension.clone(),
|
||||
};
|
||||
let terminal_event = if was_cancelled {
|
||||
crate::ingest_progress::IngestEvent::Aborted {
|
||||
@@ -654,6 +675,7 @@ pub fn ingest_with_config_opts(
|
||||
unchanged: unchanged_count,
|
||||
errors: error_count,
|
||||
duration_ms,
|
||||
skipped_by_extension,
|
||||
items: if summary_only { None } else { Some(items) },
|
||||
})
|
||||
}
|
||||
@@ -813,6 +835,31 @@ fn try_skip_unchanged(
|
||||
}))
|
||||
}
|
||||
|
||||
/// p9-fb-25: extract the lowercase extension (no leading dot) from a
|
||||
/// workspace path for use in the `unsupported media type: .X` warning
|
||||
/// and `IngestReport.skipped_by_extension` key. Returns [`NO_EXT_SENTINEL`]
|
||||
/// for paths with no extension. Always lowercase so `Foo.DOCX` and
|
||||
/// `bar.docx` aggregate under the same key.
|
||||
fn ext_for_skip_warning(path: &str) -> String {
|
||||
std::path::Path::new(path)
|
||||
.extension()
|
||||
.and_then(|s| s.to_str())
|
||||
.map(|s| s.to_ascii_lowercase())
|
||||
.unwrap_or_else(|| NO_EXT_SENTINEL.to_string())
|
||||
}
|
||||
|
||||
/// p9-fb-25: render the `IngestItem.warnings` line for a Skipped
|
||||
/// asset. [`NO_EXT_SENTINEL`] renders without a leading dot;
|
||||
/// everything else gets `.ext` form.
|
||||
fn unsupported_media_warning(path: &str) -> String {
|
||||
let ext = ext_for_skip_warning(path);
|
||||
if ext == NO_EXT_SENTINEL {
|
||||
format!("unsupported media type: {NO_EXT_SENTINEL}")
|
||||
} else {
|
||||
format!("unsupported media type: .{ext}")
|
||||
}
|
||||
}
|
||||
|
||||
/// Process a single asset: read bytes, parse, normalize, chunk,
|
||||
/// persist, embed. Per-asset failures bubble up to the caller for
|
||||
/// labelling as `IngestItemKind::Error` — they do NOT abort the
|
||||
@@ -876,7 +923,7 @@ fn ingest_one_asset(
|
||||
chunk_count: None,
|
||||
parser_version: None,
|
||||
chunker_version: None,
|
||||
warnings: Vec::new(),
|
||||
warnings: vec![unsupported_media_warning(&asset.workspace_path.0)],
|
||||
error: None,
|
||||
});
|
||||
}
|
||||
@@ -895,9 +942,7 @@ fn ingest_one_asset(
|
||||
chunk_count: None,
|
||||
parser_version: None,
|
||||
chunker_version: None,
|
||||
warnings: vec![
|
||||
"kb:// source URIs are not supported by the fs ingester".into(),
|
||||
],
|
||||
warnings: vec!["kb:// URI not yet supported".to_string()],
|
||||
error: None,
|
||||
});
|
||||
}
|
||||
@@ -1090,7 +1135,7 @@ fn ingest_one_image_asset(
|
||||
parser_version: None,
|
||||
chunker_version: None,
|
||||
warnings: vec![
|
||||
"kb:// source URIs are not supported by the fs ingester".into(),
|
||||
"kb:// URI not yet supported".to_string(),
|
||||
],
|
||||
error: None,
|
||||
});
|
||||
@@ -1425,7 +1470,7 @@ fn ingest_one_pdf_asset(
|
||||
parser_version: None,
|
||||
chunker_version: None,
|
||||
warnings: vec![
|
||||
"kb:// source URIs are not supported by the fs ingester".into(),
|
||||
"kb:// URI not yet supported".to_string(),
|
||||
],
|
||||
error: None,
|
||||
});
|
||||
|
||||
@@ -75,8 +75,8 @@ impl TestEnv {
|
||||
pub fn scope(&self) -> kebab_core::SourceScope {
|
||||
kebab_core::SourceScope {
|
||||
root: self.workspace_root.clone(),
|
||||
include: self.config.workspace.include.clone(),
|
||||
exclude: self.config.workspace.exclude.clone(),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -33,10 +33,8 @@ fn write_red_png(root: &Path, name: &str) -> std::path::PathBuf {
|
||||
|
||||
fn cfg_with_image_pipeline(env: &TestEnv, mock_endpoint: &str) -> Config {
|
||||
let mut cfg = env.config.clone();
|
||||
// Ensure image assets are scanned.
|
||||
cfg.workspace
|
||||
.include
|
||||
.push("**/*.png".to_string());
|
||||
// p9-fb-25: workspace.include removed; extension routing is now
|
||||
// handled by extractor matching alone (no config knob).
|
||||
cfg.image.ocr.enabled = true;
|
||||
cfg.image.ocr.endpoint = Some(mock_endpoint.to_string());
|
||||
cfg.image.ocr.model = "vision-mock:1b".to_string();
|
||||
@@ -261,7 +259,8 @@ async fn image_indexed_with_filename_when_ocr_and_caption_disabled() {
|
||||
let env = TestEnv::lexical_only();
|
||||
write_red_png(&env.workspace_root, "raw.png");
|
||||
let mut cfg = env.config.clone();
|
||||
cfg.workspace.include.push("**/*.png".to_string());
|
||||
// p9-fb-25: workspace.include removed; extension routing is now
|
||||
// handled by extractor matching alone (no config knob).
|
||||
cfg.image.ocr.enabled = false;
|
||||
cfg.image.caption.enabled = false;
|
||||
|
||||
@@ -326,7 +325,8 @@ async fn garbage_png_increments_errors_counter_exactly_once() {
|
||||
)
|
||||
.expect("write garbage fixture");
|
||||
let mut cfg = env.config.clone();
|
||||
cfg.workspace.include.push("**/*.png".to_string());
|
||||
// p9-fb-25: workspace.include removed; extension routing is now
|
||||
// handled by extractor matching alone (no config knob).
|
||||
cfg.image.ocr.enabled = false;
|
||||
cfg.image.caption.enabled = false;
|
||||
|
||||
|
||||
34
crates/kebab-app/tests/init_template.rs
Normal file
34
crates/kebab-app/tests/init_template.rs
Normal file
@@ -0,0 +1,34 @@
|
||||
//! p9-fb-25 task 3: `kebab init` produces config.toml with a header
|
||||
//! comment listing the four supported extensions (md / png / jpg+jpeg
|
||||
//! / pdf) so a user editing the config knows what's processable.
|
||||
|
||||
#[test]
|
||||
fn init_workspace_header_lists_supported_extensions() {
|
||||
let tmp = tempfile::tempdir().expect("tempdir");
|
||||
// SAFETY: Rust 2024 marks set_var as unsafe — wrap in unsafe block.
|
||||
// Each test sets process-wide XDG_CONFIG_HOME to point at the
|
||||
// tempdir; init_workspace writes config.toml relative to it.
|
||||
unsafe {
|
||||
std::env::set_var("XDG_CONFIG_HOME", tmp.path());
|
||||
// Same dir for data + cache to avoid touching real user paths.
|
||||
std::env::set_var("XDG_DATA_HOME", tmp.path().join("data"));
|
||||
std::env::set_var("XDG_CACHE_HOME", tmp.path().join("cache"));
|
||||
std::env::set_var("XDG_STATE_HOME", tmp.path().join("state"));
|
||||
}
|
||||
kebab_app::init_workspace(true).expect("init_workspace");
|
||||
let cfg_path = kebab_config::Config::xdg_config_path();
|
||||
let body = std::fs::read_to_string(&cfg_path).unwrap_or_else(|e| {
|
||||
panic!("read config at {}: {e}", cfg_path.display())
|
||||
});
|
||||
assert!(
|
||||
body.contains("처리 가능한 형식"),
|
||||
"header lists supported types section: body=\n{body}"
|
||||
);
|
||||
assert!(body.contains("Markdown: .md"), "md listed");
|
||||
assert!(body.contains(".png .jpg .jpeg"), "image extensions listed");
|
||||
assert!(body.contains("PDF: .pdf"), "pdf listed");
|
||||
assert!(
|
||||
!body.contains("workspace.include"),
|
||||
"no leftover include reference"
|
||||
);
|
||||
}
|
||||
@@ -121,7 +121,8 @@ fn write_pdf(root: &Path, name: &str, bytes: &[u8]) -> std::path::PathBuf {
|
||||
|
||||
fn cfg_with_pdf(env: &TestEnv) -> Config {
|
||||
let mut cfg = env.config.clone();
|
||||
cfg.workspace.include.push("**/*.pdf".to_string());
|
||||
// p9-fb-25: workspace.include removed; extension routing is now
|
||||
// handled by extractor matching alone (no config knob).
|
||||
// PDF ingest does not need OCR / caption / LM — leave defaults
|
||||
// (ocr.enabled=false, caption.enabled=false). The image pipeline
|
||||
// construction step skips both adapters.
|
||||
|
||||
43
crates/kebab-app/tests/skip_reason.rs
Normal file
43
crates/kebab-app/tests/skip_reason.rs
Normal file
@@ -0,0 +1,43 @@
|
||||
//! p9-fb-25 task 5: skipped per-asset items must carry a human-readable
|
||||
//! reason in `warnings`, and the report's `skipped_by_extension` must
|
||||
//! aggregate by lowercase extension.
|
||||
|
||||
mod common;
|
||||
|
||||
use common::TestEnv;
|
||||
|
||||
#[test]
|
||||
fn unsupported_extension_skip_carries_warning_and_is_aggregated() {
|
||||
let env = TestEnv::lexical_only();
|
||||
let workspace_root = std::path::PathBuf::from(&env.config.workspace.root);
|
||||
std::fs::write(workspace_root.join("legacy.docx"), b"unsupported").unwrap();
|
||||
std::fs::write(workspace_root.join("Makefile"), b"unsupported").unwrap();
|
||||
|
||||
let report = kebab_app::ingest_with_config(
|
||||
env.config.clone(),
|
||||
env.scope(),
|
||||
false,
|
||||
).unwrap();
|
||||
|
||||
let items = report.items.as_ref().expect("items array populated");
|
||||
let docx_item = items
|
||||
.iter()
|
||||
.find(|i| i.doc_path.0.ends_with("legacy.docx"))
|
||||
.expect("docx in items");
|
||||
assert_eq!(docx_item.kind, kebab_core::IngestItemKind::Skipped);
|
||||
assert_eq!(
|
||||
docx_item.warnings,
|
||||
vec!["unsupported media type: .docx".to_string()],
|
||||
);
|
||||
let makefile_item = items
|
||||
.iter()
|
||||
.find(|i| i.doc_path.0.ends_with("Makefile"))
|
||||
.expect("Makefile in items");
|
||||
assert_eq!(makefile_item.kind, kebab_core::IngestItemKind::Skipped);
|
||||
assert_eq!(
|
||||
makefile_item.warnings,
|
||||
vec!["unsupported media type: <no-ext>".to_string()],
|
||||
);
|
||||
assert_eq!(report.skipped_by_extension.get("docx").copied(), Some(1));
|
||||
assert_eq!(report.skipped_by_extension.get("<no-ext>").copied(), Some(1));
|
||||
}
|
||||
@@ -326,8 +326,8 @@ fn run(cli: &Cli) -> anyhow::Result<()> {
|
||||
let cfg = kebab_config::Config::load(cli.config.as_deref())?;
|
||||
let scope = kebab_core::SourceScope {
|
||||
root: root.clone().unwrap_or_else(|| PathBuf::from(&cfg.workspace.root)),
|
||||
include: cfg.workspace.include.clone(),
|
||||
exclude: cfg.workspace.exclude.clone(),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
// p9-fb-02: spawn the progress display on a background
|
||||
@@ -371,12 +371,14 @@ fn run(cli: &Cli) -> anyhow::Result<()> {
|
||||
if cli.json {
|
||||
println!("{}", serde_json::to_string(&wire::wire_ingest(&report))?);
|
||||
} else {
|
||||
let skipped_breakdown = kebab_app::render_skipped_breakdown(&report.skipped_by_extension);
|
||||
println!(
|
||||
"scanned {} new {} updated {} skipped {} errors {} ({} ms)",
|
||||
"scanned {} new {} updated {} skipped {}{} errors {} ({} ms)",
|
||||
report.scanned,
|
||||
report.new,
|
||||
report.updated,
|
||||
report.skipped,
|
||||
skipped_breakdown,
|
||||
report.errors,
|
||||
report.duration_ms
|
||||
);
|
||||
|
||||
@@ -171,6 +171,7 @@ mod tests {
|
||||
unchanged: 0,
|
||||
errors: 0,
|
||||
duration_ms: 0,
|
||||
skipped_by_extension: std::collections::BTreeMap::new(),
|
||||
items: None,
|
||||
};
|
||||
let v = wire_ingest(&r);
|
||||
|
||||
@@ -51,7 +51,6 @@ pub struct Config {
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct WorkspaceCfg {
|
||||
pub root: String,
|
||||
pub include: Vec<String>,
|
||||
pub exclude: Vec<String>,
|
||||
}
|
||||
|
||||
@@ -251,7 +250,6 @@ impl Config {
|
||||
schema_version: 1,
|
||||
workspace: WorkspaceCfg {
|
||||
root: "~/KnowledgeBase".to_string(),
|
||||
include: vec!["**/*.md".to_string()],
|
||||
exclude: vec![
|
||||
".git/**".to_string(),
|
||||
"node_modules/**".to_string(),
|
||||
@@ -396,6 +394,29 @@ impl Config {
|
||||
/// than the user's `cwd`.
|
||||
pub fn from_file(path: &Path) -> anyhow::Result<Self> {
|
||||
let text = std::fs::read_to_string(path)?;
|
||||
|
||||
// p9-fb-25: probe for the legacy `workspace.include` key — if
|
||||
// present, emit a one-shot deprecation warning. Detection uses
|
||||
// raw `toml::Value` lookup; the warning fires via a process-
|
||||
// level OnceLock so a long-running TUI / CLI run doesn't spam
|
||||
// the log on every Config::load.
|
||||
if let Ok(value) = toml::from_str::<toml::Value>(&text) {
|
||||
if value
|
||||
.get("workspace")
|
||||
.and_then(|v| v.get("include"))
|
||||
.is_some()
|
||||
{
|
||||
static DEPRECATION_FIRED: std::sync::OnceLock<()> = std::sync::OnceLock::new();
|
||||
DEPRECATION_FIRED.get_or_init(|| {
|
||||
tracing::warn!(
|
||||
target: "kebab-config",
|
||||
config = %path.display(),
|
||||
"deprecated config: `workspace.include` 필드는 더 이상 사용되지 않습니다 (p9-fb-25, v0.2.1+). 처리 가능한 형식 (md / png / jpg / pdf) 은 extractor 가 자동 결정. config 에서 이 필드를 제거해도 안전 — 더 이상 enforce 안 됨."
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
let mut cfg: Self = toml::from_str(&text)?;
|
||||
cfg.source_dir = path.parent().map(Path::to_path_buf);
|
||||
Ok(cfg)
|
||||
@@ -868,6 +889,32 @@ max_context_tokens = 8000
|
||||
assert_eq!(c.image, ImageCfg::defaults());
|
||||
}
|
||||
|
||||
/// p9-fb-25: legacy config with `workspace.include = [...]` must
|
||||
/// still deserialize cleanly (silent unknown-field acceptance).
|
||||
#[test]
|
||||
fn legacy_include_field_is_ignored_silently() {
|
||||
let mut cfg = Config::defaults();
|
||||
cfg.workspace.root = "/tmp/kebab-legacy".to_string();
|
||||
let mut toml_text = toml::to_string(&cfg).expect("default round-trips");
|
||||
// Inject a legacy `include = [...]` line into the [workspace] block.
|
||||
toml_text = toml_text.replace(
|
||||
"[workspace]",
|
||||
"[workspace]\ninclude = [\"**/*.md\", \"**/*.txt\"]",
|
||||
);
|
||||
let parsed: Result<Config, _> = toml::from_str(&toml_text);
|
||||
assert!(parsed.is_ok(), "legacy include must not break load: {:?}", parsed.err());
|
||||
let cfg = parsed.unwrap();
|
||||
assert_eq!(cfg.workspace.root, "/tmp/kebab-legacy");
|
||||
}
|
||||
|
||||
/// p9-fb-25: `WorkspaceCfg` must NOT have an `include` field.
|
||||
/// Compile-time proof: exhaustive destructure.
|
||||
#[test]
|
||||
fn workspace_cfg_has_only_root_and_exclude_fields() {
|
||||
let ws = Config::defaults().workspace;
|
||||
let WorkspaceCfg { root: _, exclude: _ } = &ws;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn xdg_paths_honor_env() {
|
||||
// Must restore env after the test to avoid polluting other tests.
|
||||
|
||||
@@ -20,6 +20,11 @@ pub struct IngestReport {
|
||||
pub unchanged: u32,
|
||||
pub errors: u32,
|
||||
pub duration_ms: u32,
|
||||
/// p9-fb-25: per-extension skip count. Key = lowercase extension
|
||||
/// without leading dot (e.g. "docx", "txt"); files without an
|
||||
/// extension key under "<no-ext>". `BTreeMap` so the wire JSON
|
||||
/// has stable key order across runs.
|
||||
pub skipped_by_extension: std::collections::BTreeMap<String, u32>,
|
||||
/// `None` ↔ wire `items: null` (`--summary-only`).
|
||||
pub items: Option<Vec<IngestItem>>,
|
||||
}
|
||||
|
||||
@@ -43,6 +43,7 @@
|
||||
"root": "/home/u/KB"
|
||||
},
|
||||
"skipped": 0,
|
||||
"skipped_by_extension": {},
|
||||
"unchanged": 0,
|
||||
"updated": 1
|
||||
}
|
||||
|
||||
@@ -34,6 +34,7 @@ fn fixture_report() -> IngestReport {
|
||||
unchanged: 0,
|
||||
errors: 0,
|
||||
duration_ms: 187,
|
||||
skipped_by_extension: std::collections::BTreeMap::new(),
|
||||
items: Some(vec![
|
||||
IngestItem {
|
||||
kind: IngestItemKind::New,
|
||||
|
||||
@@ -36,8 +36,8 @@ pub fn start_ingest(app: &mut App) -> anyhow::Result<()> {
|
||||
let cfg = app.config.clone();
|
||||
let scope = SourceScope {
|
||||
root: std::path::PathBuf::from(&cfg.workspace.root),
|
||||
include: cfg.workspace.include.clone(),
|
||||
exclude: cfg.workspace.exclude.clone(),
|
||||
..Default::default()
|
||||
};
|
||||
let (tx, rx) = mpsc::channel::<IngestEvent>();
|
||||
let cancel = Arc::new(AtomicBool::new(false));
|
||||
@@ -175,8 +175,9 @@ pub fn status_line(state: &IngestState) -> String {
|
||||
let elapsed = state.started_at.elapsed();
|
||||
let secs = elapsed.as_secs();
|
||||
if state.aborted {
|
||||
let skipped_breakdown = kebab_app::ingest_progress::render_skipped_breakdown(&state.counts.skipped_by_extension);
|
||||
return format!(
|
||||
"✗ ingest aborted at {}/{} after {}s (new={} updated={} unchanged={} skipped={} errors={})",
|
||||
"✗ ingest aborted at {}/{} after {}s (new={} updated={} unchanged={} skipped={}{} errors={})",
|
||||
state.counts.scanned.saturating_sub(state.counts.errors),
|
||||
state.counts.scanned,
|
||||
secs,
|
||||
@@ -184,16 +185,19 @@ pub fn status_line(state: &IngestState) -> String {
|
||||
state.counts.updated,
|
||||
state.counts.unchanged,
|
||||
state.counts.skipped,
|
||||
skipped_breakdown,
|
||||
state.counts.errors,
|
||||
);
|
||||
}
|
||||
let skipped_breakdown = kebab_app::ingest_progress::render_skipped_breakdown(&state.counts.skipped_by_extension);
|
||||
return format!(
|
||||
"✓ ingest: {} docs ({} new, {} updated, {} unchanged, {} skipped), {} chunks indexed in {}s",
|
||||
"✓ ingest: {} docs ({} new, {} updated, {} unchanged, {} skipped{}), {} chunks indexed in {}s",
|
||||
state.counts.scanned,
|
||||
state.counts.new,
|
||||
state.counts.updated,
|
||||
state.counts.unchanged,
|
||||
state.counts.skipped,
|
||||
skipped_breakdown,
|
||||
state.counts.chunks_indexed,
|
||||
secs,
|
||||
);
|
||||
@@ -288,7 +292,7 @@ mod tests {
|
||||
chunks_indexed: 50,
|
||||
..Default::default()
|
||||
};
|
||||
apply_event(&mut s, IngestEvent::Completed { counts: final_counts });
|
||||
apply_event(&mut s, IngestEvent::Completed { counts: final_counts.clone() });
|
||||
assert_eq!(s.counts, final_counts);
|
||||
assert!(s.terminal_at.is_some());
|
||||
assert!(!s.aborted);
|
||||
@@ -415,4 +419,44 @@ mod tests {
|
||||
// No worker to cancel — already terminated.
|
||||
assert!(!cancel_running_ingest(&app));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn status_line_terminal_includes_skipped_breakdown() {
|
||||
let mut s = fresh_state();
|
||||
let skipped_by_extension = std::collections::BTreeMap::from([
|
||||
("docx".to_string(), 2u32),
|
||||
("txt".to_string(), 1u32),
|
||||
]);
|
||||
let counts = AggregateCounts {
|
||||
scanned: 10,
|
||||
skipped: 3,
|
||||
skipped_by_extension,
|
||||
..Default::default()
|
||||
};
|
||||
apply_event(&mut s, IngestEvent::Completed { counts });
|
||||
let line = status_line(&s);
|
||||
assert!(
|
||||
line.contains("3 skipped: 2 docx, 1 txt"),
|
||||
"breakdown must appear in: {line}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn status_line_aborted_includes_skipped_breakdown() {
|
||||
let mut s = fresh_state();
|
||||
let skipped_by_extension =
|
||||
std::collections::BTreeMap::from([("pdf".to_string(), 2u32)]);
|
||||
let counts = AggregateCounts {
|
||||
scanned: 5,
|
||||
skipped: 2,
|
||||
skipped_by_extension,
|
||||
..Default::default()
|
||||
};
|
||||
apply_event(&mut s, IngestEvent::Aborted { counts });
|
||||
let line = status_line(&s);
|
||||
assert!(
|
||||
line.contains("skipped=2: 2 pdf"),
|
||||
"breakdown must appear in: {line}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,882 @@
|
||||
# p9-fb-25 — Config `workspace.include` 제거 + 지원 형식 가시성 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Remove the dead `WorkspaceCfg.include` config field, surface skip reasons (unsupported media type) per file via `IngestItem.warnings`, and aggregate them into `IngestReport.skipped_by_extension` with CLI / TUI / README / `kebab init` template all calling out the four supported extensions (`.md`, `.png`, `.jpg/.jpeg`, `.pdf`).
|
||||
|
||||
**Architecture:** Two coordinated streams. (1) Config: drop `WorkspaceCfg.include` + emit a one-shot `tracing::warn!` when an old config still has it (raw TOML key probe). (2) Ingest pipeline: every Skipped per-asset return gains `warnings: vec!["unsupported media type: .ext"]` (or `"kb:// URI not yet supported"`); the asset loop bumps a new `aggregate.skipped_by_extension: BTreeMap<String, u32>` keyed by lowercase ext (`<no-ext>` sentinel for files without one). CLI summary + TUI status_line render the breakdown desc-sorted on terminal events.
|
||||
|
||||
**Tech Stack:** Rust 2024, serde, toml 0.8, tracing. `BTreeMap` for stable JSON key order. No new deps. No SQLite migration.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-05-05-p9-fb-25-config-include-removal-design.md`
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
**Modified:**
|
||||
- `crates/kebab-config/src/lib.rs` — drop `WorkspaceCfg.include` field + update `WorkspaceCfg::defaults()` + add `from_file` deprecation probe.
|
||||
- `crates/kebab-app/src/lib.rs` — `init_workspace` header comment lists supported extensions + (no code change needed beyond default config no longer carrying include); ingest pipeline's three Skipped emit sites populate `warnings` + bump `skipped_by_extension`.
|
||||
- `crates/kebab-app/src/ingest_progress.rs` — `AggregateCounts.skipped_by_extension: BTreeMap<String, u32>`.
|
||||
- `crates/kebab-core/src/ingest.rs` — `IngestReport.skipped_by_extension: BTreeMap<String, u32>`.
|
||||
- `crates/kebab-cli/src/main.rs` — drop `include: cfg.workspace.include.clone()` from SourceScope construction; render breakdown in summary print.
|
||||
- `crates/kebab-tui/src/ingest_progress.rs` — drop `include: cfg.workspace.include.clone()` from SourceScope construction; render breakdown in `status_line` final / aborted.
|
||||
- `docs/wire-schema/v1/ingest_report.schema.json` — additive `skipped_by_extension` (object, additionalProperties integer ≥ 0).
|
||||
- `README.md` — `kebab tui` / `kebab ingest` cell appends supported-extension list + skip-reason mention.
|
||||
- `HANDOFF.md`, `tasks/HOTFIXES.md`, `tasks/INDEX.md`, `tasks/p9/p9-fb-25-config-include-removal.md`.
|
||||
|
||||
`SourceScope` (in `kebab-core/src/traits.rs`) keeps its `include: Vec<String>` field — it's a design-level abstraction (§7.1) that connectors / routers may use later. Removing it is a separate spec.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Drop `WorkspaceCfg.include` + add deprecation probe
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/kebab-config/src/lib.rs` (struct + defaults + from_file)
|
||||
|
||||
This task removes the field but keeps backward-compat: an old `config.toml` with `include = [...]` still loads (serde ignores unknown keys without `deny_unknown_fields`). On detection, emit a one-shot `tracing::warn!`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
Append to the existing `#[cfg(test)] mod tests` block in `crates/kebab-config/src/lib.rs` (find via `grep -n "fn defaults_are_serde_roundtrip_stable" crates/kebab-config/src/lib.rs`):
|
||||
|
||||
```rust
|
||||
/// p9-fb-25: legacy config with `workspace.include = [...]` must
|
||||
/// still deserialize cleanly (silent unknown-field acceptance).
|
||||
#[test]
|
||||
fn legacy_include_field_is_ignored_silently() {
|
||||
let toml_text = r#"
|
||||
schema_version = 1
|
||||
|
||||
[workspace]
|
||||
root = "/tmp/kebab-legacy"
|
||||
include = ["**/*.md", "**/*.txt"]
|
||||
exclude = [".git/**"]
|
||||
|
||||
[storage]
|
||||
data_dir = "/tmp/kebab-data"
|
||||
sqlite = "{data_dir}/kebab.sqlite"
|
||||
vector_dir = "{data_dir}/lancedb"
|
||||
asset_dir = "{data_dir}/assets"
|
||||
artifact_dir = "{data_dir}/artifacts"
|
||||
model_dir = "{data_dir}/models"
|
||||
runs_dir = "{data_dir}/runs"
|
||||
copy_threshold_mb = 100
|
||||
|
||||
[indexing]
|
||||
max_parallel_extractors = 2
|
||||
max_parallel_embeddings = 1
|
||||
watch_filesystem = false
|
||||
"#;
|
||||
// NOTE: a real legacy config has many more sections (chunking,
|
||||
// models, etc.). For this test we rely on `#[serde(default)]`
|
||||
// on each top-level Config field — if any field is missing
|
||||
// a serde default at this point, we accept that as a separate
|
||||
// bug and adjust below. The point of THIS test is the
|
||||
// workspace.include field tolerance.
|
||||
let parsed: Result<Config, _> = toml::from_str(toml_text);
|
||||
assert!(parsed.is_ok(), "legacy include must not break load: {:?}", parsed.err());
|
||||
let cfg = parsed.unwrap();
|
||||
assert_eq!(cfg.workspace.root, "/tmp/kebab-legacy");
|
||||
assert_eq!(cfg.workspace.exclude, vec![".git/**".to_string()]);
|
||||
}
|
||||
|
||||
/// p9-fb-25: `WorkspaceCfg::defaults()` no longer carries `include`.
|
||||
#[test]
|
||||
fn workspace_defaults_have_no_include_field() {
|
||||
let ws = WorkspaceCfg::defaults();
|
||||
// We're not asserting include is absent (Rust struct field
|
||||
// doesn't have such a query). We assert the default has the
|
||||
// expected exclude shape and the struct definition reflects
|
||||
// the removal — this test will fail to compile if a stray
|
||||
// reference to ws.include lingers.
|
||||
assert!(!ws.exclude.is_empty(), "default exclude should retain `.git/**` etc.");
|
||||
}
|
||||
```
|
||||
|
||||
If `Config` doesn't have `#[serde(default)]` at the section level (chunking / models / etc.), the legacy-config test will fail because the abbreviated TOML omits required sections. In that case, expand the legacy TOML in the test to include all required sections — the goal is to verify `include` is ignored, not to test default fallback. Use `Config::defaults()` and serialize → modify → deserialize as a shortcut:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn legacy_include_field_is_ignored_silently() {
|
||||
let mut cfg = Config::defaults();
|
||||
cfg.workspace.root = "/tmp/kebab-legacy".to_string();
|
||||
let mut toml_text = toml::to_string(&cfg).expect("default round-trips");
|
||||
// Inject a legacy `include = [...]` line into the [workspace] block.
|
||||
toml_text = toml_text.replace(
|
||||
"[workspace]",
|
||||
"[workspace]\ninclude = [\"**/*.md\", \"**/*.txt\"]",
|
||||
);
|
||||
let parsed: Result<Config, _> = toml::from_str(&toml_text);
|
||||
assert!(parsed.is_ok(), "legacy include must not break load: {:?}", parsed.err());
|
||||
let cfg = parsed.unwrap();
|
||||
assert_eq!(cfg.workspace.root, "/tmp/kebab-legacy");
|
||||
}
|
||||
```
|
||||
|
||||
Run: `cargo test -p kebab-config --lib legacy_include_field_is_ignored_silently`
|
||||
Expected: FAIL — current `WorkspaceCfg` still has `include: Vec<String>` so deserialize SUCCEEDS but `workspace_defaults_have_no_include_field` test compiles and passes; the first test passes too because serde default for `Vec<String>` is empty. Wait — the FAIL precondition is wrong. Both tests pass against current code.
|
||||
|
||||
Reframe: write a **forward-looking** test that asserts the field is gone:
|
||||
|
||||
```rust
|
||||
/// p9-fb-25: `WorkspaceCfg` must NOT have an `include` field.
|
||||
/// Compile-time proof: this test references every field of
|
||||
/// `WorkspaceCfg` exhaustively. If a future commit re-introduces
|
||||
/// `include`, the destructure here breaks (refactor failure).
|
||||
#[test]
|
||||
fn workspace_cfg_has_only_root_and_exclude_fields() {
|
||||
let ws = WorkspaceCfg::defaults();
|
||||
// Exhaustive destructure — adding a new field would break
|
||||
// this on the next compile.
|
||||
let WorkspaceCfg { root: _, exclude: _ } = &ws;
|
||||
}
|
||||
```
|
||||
|
||||
This test will NOT compile against current code (because `WorkspaceCfg` still has `include`). The compile error IS the test failure.
|
||||
|
||||
Run: `cargo build -p kebab-config --tests`
|
||||
Expected: error[E0027] missing structure fields OR error mentioning `include`. **This is the failing test.**
|
||||
|
||||
- [ ] **Step 2: Drop the field + update defaults**
|
||||
|
||||
Open `crates/kebab-config/src/lib.rs`. Find `pub struct WorkspaceCfg` (around line 51). Remove the `pub include: Vec<String>` line:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct WorkspaceCfg {
|
||||
pub root: String,
|
||||
pub exclude: Vec<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Find `WorkspaceCfg::defaults()` or the `Config::defaults()` body that constructs `WorkspaceCfg { root, include, exclude }` (around line 252). Drop the `include: vec!["**/*.md".to_string()],` line:
|
||||
|
||||
```rust
|
||||
workspace: WorkspaceCfg {
|
||||
root: "~/KnowledgeBase".to_string(),
|
||||
exclude: vec![
|
||||
".git/**".to_string(),
|
||||
"node_modules/**".to_string(),
|
||||
".obsidian/**".to_string(),
|
||||
],
|
||||
},
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add deprecation probe in `from_file`**
|
||||
|
||||
In the same file, find `pub fn from_file` (around line 397). Replace the body to probe for the legacy `include` key BEFORE typed deserialize:
|
||||
|
||||
```rust
|
||||
pub fn from_file(path: &Path) -> anyhow::Result<Self> {
|
||||
let text = std::fs::read_to_string(path)?;
|
||||
|
||||
// p9-fb-25: probe for the legacy `workspace.include` key — if
|
||||
// present, emit a one-shot deprecation warning. Detection uses
|
||||
// raw `toml::Value` lookup; the warning is fired via a
|
||||
// process-level `OnceLock` so a long-running TUI / CLI run
|
||||
// doesn't spam the log on every Config::load.
|
||||
if let Ok(value) = toml::from_str::<toml::Value>(&text) {
|
||||
if value
|
||||
.get("workspace")
|
||||
.and_then(|v| v.get("include"))
|
||||
.is_some()
|
||||
{
|
||||
static DEPRECATION_FIRED: std::sync::OnceLock<()> = std::sync::OnceLock::new();
|
||||
DEPRECATION_FIRED.get_or_init(|| {
|
||||
tracing::warn!(
|
||||
target: "kebab-config",
|
||||
config = %path.display(),
|
||||
"deprecated config: `workspace.include` 필드는 더 이상 사용되지 않습니다 (p9-fb-25). 처리 가능한 형식 (md / png / jpg / pdf) 은 extractor 가 자동 결정. 다음 버전부터 config 갱신 권장."
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
let mut cfg: Self = toml::from_str(&text)?;
|
||||
cfg.source_dir = path.parent().map(Path::to_path_buf);
|
||||
Ok(cfg)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests**
|
||||
|
||||
Run: `cargo test -p kebab-config --lib`
|
||||
Expected: all pass. The forward-looking exhaustive-destructure test now compiles + asserts the struct shape.
|
||||
|
||||
- [ ] **Step 5: Build the workspace + surface compile errors**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
Expected: compile errors at every site that constructs `WorkspaceCfg { ..., include: ..., ... }` or reads `cfg.workspace.include`. The known sites:
|
||||
- `crates/kebab-cli/src/main.rs:329` — drop `include: cfg.workspace.include.clone(),` (Task 2 will add the proper SourceScope construction).
|
||||
- `crates/kebab-tui/src/ingest_progress.rs:39` — same.
|
||||
- Test fixtures inside `kebab-config` if any — drop the `include: vec![...]` literal.
|
||||
|
||||
For Task 1's commit, fix ONLY the kebab-config sites (the test passes fix). Other crates' compile errors roll into Task 2.
|
||||
|
||||
For now, in `crates/kebab-cli/src/main.rs` line 329 and `crates/kebab-tui/src/ingest_progress.rs` line 39, replace `include: cfg.workspace.include.clone()` with `include: Vec::new()` (Task 2 will use `..Default::default()` once we touch the structures more carefully).
|
||||
|
||||
- [ ] **Step 6: clippy + commit**
|
||||
|
||||
Run: `cargo clippy -p kebab-config --all-targets -- -D warnings`
|
||||
Expected: clean.
|
||||
|
||||
```bash
|
||||
git add crates/kebab-config/src/lib.rs crates/kebab-cli/src/main.rs crates/kebab-tui/src/ingest_progress.rs
|
||||
git commit -m "feat(kebab-config): p9-fb-25 task 1 — drop WorkspaceCfg.include + deprecation probe
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Switch SourceScope construction to `..Default::default()` for cleaner removal
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/kebab-cli/src/main.rs` (line 327)
|
||||
- Modify: `crates/kebab-tui/src/ingest_progress.rs` (line 38)
|
||||
|
||||
Task 1's quick fix (`include: Vec::new()`) is functional but ugly. This task replaces those with the idiomatic `..Default::default()` pattern.
|
||||
|
||||
- [ ] **Step 1: Update CLI ingest dispatch**
|
||||
|
||||
Open `crates/kebab-cli/src/main.rs`. Find the `Cmd::Ingest { ... }` arm (around line 321). Replace the SourceScope literal:
|
||||
|
||||
```rust
|
||||
let scope = kebab_core::SourceScope {
|
||||
root: root.clone().unwrap_or_else(|| PathBuf::from(&cfg.workspace.root)),
|
||||
exclude: cfg.workspace.exclude.clone(),
|
||||
..Default::default()
|
||||
};
|
||||
```
|
||||
|
||||
(SourceScope derives `Default` per `kebab-core/src/traits.rs`; `..Default::default()` fills `include: Vec::new()` for now. If `include` is removed from `SourceScope` in a future spec, this site needs no change.)
|
||||
|
||||
- [ ] **Step 2: Update TUI ingest dispatch**
|
||||
|
||||
Open `crates/kebab-tui/src/ingest_progress.rs`. Find the SourceScope construction (around line 38):
|
||||
|
||||
```rust
|
||||
scope: kebab_core::SourceScope {
|
||||
root: PathBuf::from(&cfg.workspace.root),
|
||||
exclude: cfg.workspace.exclude.clone(),
|
||||
..Default::default()
|
||||
},
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Build + test**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
Run: `cargo test -p kebab-cli -p kebab-tui --lib`
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all clean.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/kebab-cli/src/main.rs crates/kebab-tui/src/ingest_progress.rs
|
||||
git commit -m "refactor(kebab-cli, kebab-tui): p9-fb-25 task 2 — SourceScope via ..Default::default()
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: `init_workspace` header — supported extensions
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/kebab-app/src/lib.rs` (`init_workspace`, around line 138)
|
||||
|
||||
- [ ] **Step 1: Update the header comment**
|
||||
|
||||
Open `crates/kebab-app/src/lib.rs`. Find `let header = "\` inside `init_workspace` (around line 138). Replace the entire `header` string with a version that adds the supported-extensions block AND removes any reference to `include`:
|
||||
|
||||
```rust
|
||||
let header = "\
|
||||
# kebab config — `~/.config/kebab/config.toml`.
|
||||
#
|
||||
# `workspace.root` accepts:
|
||||
# • absolute paths (`/home/me/KnowledgeBase`)
|
||||
# • tilde (`~/KnowledgeBase`) ← default
|
||||
# • env vars (`${XDG_DATA_HOME}/kebab`)
|
||||
# • relative paths (`./notes`, `notes`, `../shared/x`)
|
||||
# — relative paths resolve against the directory of THIS
|
||||
# config file, NOT the user's `cwd` at invocation time.
|
||||
#
|
||||
# 처리 가능한 형식 (extractor 가 자동 결정 — config 에 명시할 수 없음):
|
||||
# • Markdown: .md
|
||||
# • 이미지: .png .jpg .jpeg (OCR + caption)
|
||||
# • PDF: .pdf
|
||||
# 다른 확장자는 ingest 시 자동 skip + warning. 처리 대상 폴더의
|
||||
# 일부만 ingest 하고 싶으면 `kebab ingest <path>` 로 root 명시
|
||||
# 또는 `.kebabignore` 파일 / 본 `workspace.exclude` 로 denylist.
|
||||
#
|
||||
# Override individual keys at runtime with `KEBAB_*` env vars
|
||||
# (e.g. `KEBAB_WORKSPACE_ROOT=/tmp/test kebab ingest`).
|
||||
\n";
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Smoke test by running `kebab init` against a temp config**
|
||||
|
||||
Add (or extend) a test in `crates/kebab-app/tests/`:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn init_workspace_header_lists_supported_extensions() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
// SAFETY: each test sets KEBAB env vars in a process-wide manner;
|
||||
// this test relies on init_workspace writing relative to the
|
||||
// current XDG_CONFIG_HOME. We override XDG_CONFIG_HOME to the
|
||||
// tmp dir so the produced config sits inside `tmp`.
|
||||
unsafe {
|
||||
std::env::set_var("XDG_CONFIG_HOME", tmp.path());
|
||||
}
|
||||
kebab_app::init_workspace(true).expect("init_workspace");
|
||||
let cfg_path = kebab_config::Config::xdg_config_path();
|
||||
let body = std::fs::read_to_string(&cfg_path).unwrap();
|
||||
assert!(body.contains("처리 가능한 형식"), "header lists supported types");
|
||||
assert!(body.contains("Markdown: .md"), "md listed");
|
||||
assert!(body.contains(".png .jpg .jpeg"), "image extensions listed");
|
||||
assert!(body.contains("PDF: .pdf"), "pdf listed");
|
||||
assert!(!body.contains("workspace.include"), "no leftover include reference");
|
||||
}
|
||||
```
|
||||
|
||||
If the existing kebab-app test infra already has an `init_workspace` test, extend it. Otherwise create a new file `crates/kebab-app/tests/init_template.rs`.
|
||||
|
||||
`unsafe`: Rust 2024 + recent toolchain may flag `set_var` as unsafe. Wrap in `unsafe { ... }` block per current Rust semantics.
|
||||
|
||||
- [ ] **Step 3: Run + commit**
|
||||
|
||||
```bash
|
||||
cargo test -p kebab-app --test init_template # or whatever filename
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
git add -u
|
||||
git commit -m "feat(kebab-app): p9-fb-25 task 3 — init_workspace header lists supported extensions
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add `skipped_by_extension` to `IngestReport` + `AggregateCounts` + wire schema
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/kebab-core/src/ingest.rs`
|
||||
- Modify: `crates/kebab-app/src/ingest_progress.rs`
|
||||
- Modify: `docs/wire-schema/v1/ingest_report.schema.json`
|
||||
|
||||
- [ ] **Step 1: Add field to `IngestReport`**
|
||||
|
||||
Open `crates/kebab-core/src/ingest.rs`. Replace the `IngestReport` struct (around lines 10-22) by adding the new field:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct IngestReport {
|
||||
pub scope: SourceScope,
|
||||
pub scanned: u32,
|
||||
pub new: u32,
|
||||
pub updated: u32,
|
||||
pub skipped: u32,
|
||||
pub unchanged: u32,
|
||||
pub errors: u32,
|
||||
pub duration_ms: u32,
|
||||
/// p9-fb-25: per-extension skip count. Key = lowercase extension
|
||||
/// without leading dot (e.g. "docx", "txt"); files without an
|
||||
/// extension key under "<no-ext>". `BTreeMap` so the wire JSON
|
||||
/// has stable key order across runs.
|
||||
pub skipped_by_extension: std::collections::BTreeMap<String, u32>,
|
||||
pub items: Option<Vec<IngestItem>>,
|
||||
}
|
||||
```
|
||||
|
||||
The import for `BTreeMap` lives inside the field annotation via the full path; don't add a `use` at the top of the file.
|
||||
|
||||
- [ ] **Step 2: Add field to `AggregateCounts`**
|
||||
|
||||
Open `crates/kebab-app/src/ingest_progress.rs`. Replace the struct (around lines 25-37):
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct AggregateCounts {
|
||||
pub scanned: u32,
|
||||
pub new: u32,
|
||||
pub updated: u32,
|
||||
pub skipped: u32,
|
||||
pub unchanged: u32,
|
||||
pub errors: u32,
|
||||
pub chunks_indexed: u32,
|
||||
pub embeddings_indexed: u32,
|
||||
/// p9-fb-25: per-extension skip count. See [`IngestReport.skipped_by_extension`].
|
||||
pub skipped_by_extension: std::collections::BTreeMap<String, u32>,
|
||||
}
|
||||
```
|
||||
|
||||
Note: removed `Copy` from the derive. `BTreeMap` is not `Copy`. Replaces `Copy + Eq + PartialEq` with `Eq + PartialEq` only. `Copy`-using callers (if any) need to switch to `clone()`.
|
||||
|
||||
Run: `cargo build -p kebab-app`. Look for `error: ... requires Copy` errors. Fix each by `.clone()`.
|
||||
|
||||
- [ ] **Step 3: Update wire schema**
|
||||
|
||||
Open `docs/wire-schema/v1/ingest_report.schema.json`. Inside `properties`, add:
|
||||
|
||||
```json
|
||||
"skipped_by_extension": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"description": "p9-fb-25: per-extension skip count. Key = lowercase extension without leading dot (e.g. 'docx'). Files without extension key under '<no-ext>'."
|
||||
},
|
||||
```
|
||||
|
||||
If `required` array exists, add `skipped_by_extension` to it (always present, even if empty `{}`).
|
||||
|
||||
- [ ] **Step 4: Build + fix construction sites**
|
||||
|
||||
Run: `cargo build --workspace`. The compiler will surface every `IngestReport { ... }` and `AggregateCounts { ... }` literal that omits the new field. Add `skipped_by_extension: BTreeMap::new()` (or `Default::default()`) at each.
|
||||
|
||||
Add `use std::collections::BTreeMap;` at the top of test fixture files where the literal is constructed (so the addition is concise — `BTreeMap::new()` rather than `std::collections::BTreeMap::new()`).
|
||||
|
||||
Snapshot fixtures (`crates/kebab-store-sqlite/snapshots/ingest_report.snapshot.json`) — add `"skipped_by_extension": {}` between two existing fields (alphabetic / logical order — between `skipped` and `errors` to match struct declaration order).
|
||||
|
||||
- [ ] **Step 5: Run + commit**
|
||||
|
||||
```bash
|
||||
cargo test --workspace --no-fail-fast -j 1 2>&1 | grep "^test result:"
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
git add -u
|
||||
git commit -m "feat(kebab-core, kebab-app): p9-fb-25 task 4 — IngestReport.skipped_by_extension + wire schema additive
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Populate `IngestItem.warnings` for Skipped paths + bump `skipped_by_extension`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/kebab-app/src/lib.rs` (three Skipped emit sites + one asset loop)
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `crates/kebab-app/tests/skip_reason.rs`:
|
||||
|
||||
```rust
|
||||
//! p9-fb-25: skipped per-asset items must carry a human-readable reason
|
||||
//! in `warnings`, and the report's `skipped_by_extension` must aggregate
|
||||
//! by lowercase extension.
|
||||
|
||||
mod common;
|
||||
|
||||
use common::TestEnv;
|
||||
|
||||
#[test]
|
||||
fn unsupported_extension_skip_carries_warning_and_is_aggregated() {
|
||||
let env = TestEnv::lexical_only();
|
||||
// Workspace already populated by TestEnv; add a `.docx` and a
|
||||
// file with no extension to trigger Skipped paths.
|
||||
let workspace_root = std::path::PathBuf::from(&env.config.workspace.root);
|
||||
std::fs::write(workspace_root.join("legacy.docx"), b"unsupported").unwrap();
|
||||
std::fs::write(workspace_root.join("Makefile"), b"unsupported").unwrap();
|
||||
|
||||
let report = kebab_app::ingest_with_config(
|
||||
env.config.clone(),
|
||||
env.scope(),
|
||||
false,
|
||||
).unwrap();
|
||||
|
||||
let items = report.items.as_ref().expect("items array populated");
|
||||
let docx_item = items.iter().find(|i| i.doc_path.0.ends_with("legacy.docx")).unwrap();
|
||||
assert_eq!(docx_item.kind, kebab_core::IngestItemKind::Skipped);
|
||||
assert_eq!(
|
||||
docx_item.warnings,
|
||||
vec!["unsupported media type: .docx".to_string()],
|
||||
);
|
||||
let makefile_item = items.iter().find(|i| i.doc_path.0.ends_with("Makefile")).unwrap();
|
||||
assert_eq!(makefile_item.kind, kebab_core::IngestItemKind::Skipped);
|
||||
assert_eq!(
|
||||
makefile_item.warnings,
|
||||
vec!["unsupported media type: <no-ext>".to_string()],
|
||||
);
|
||||
assert_eq!(report.skipped_by_extension.get("docx").copied(), Some(1));
|
||||
assert_eq!(report.skipped_by_extension.get("<no-ext>").copied(), Some(1));
|
||||
}
|
||||
```
|
||||
|
||||
If `TestEnv::lexical_only()` populates a workspace with markdown fixtures already, the docx + Makefile additions are extra. If not, build a fresh `TempDir` workspace following the pattern other kebab-app tests use.
|
||||
|
||||
Run: `cargo test -p kebab-app --test skip_reason`
|
||||
Expected: FAIL — current `warnings: Vec::new()` for Skipped + `skipped_by_extension` is always empty.
|
||||
|
||||
- [ ] **Step 2: Update the three `IngestItemKind::Skipped` emit sites**
|
||||
|
||||
Open `crates/kebab-app/src/lib.rs`. Three sites currently return `IngestItem { kind: Skipped, ..., warnings: Vec::new(), ... }`:
|
||||
|
||||
- One at the top of `ingest_one_asset` (the `_ =>` fallback when MediaType doesn't match).
|
||||
- One when `SourceUri::Kb` (kb:// URI not yet supported).
|
||||
- One in `ingest_one_image_asset` when SourceUri is kb://.
|
||||
- One in `ingest_one_pdf_asset` when SourceUri is kb://.
|
||||
|
||||
For each Skipped emit, replace `warnings: Vec::new()` with the appropriate reason.
|
||||
|
||||
For the **media-type fallback** at the top of `ingest_one_asset`: extract the extension from `asset.workspace_path.0` (lowercase, no dot, `<no-ext>` sentinel) and emit:
|
||||
|
||||
```rust
|
||||
let ext = ext_for_skip_warning(&asset.workspace_path.0);
|
||||
return Ok(kebab_core::IngestItem {
|
||||
kind: kebab_core::IngestItemKind::Skipped,
|
||||
doc_id: None,
|
||||
doc_path: asset.workspace_path.clone(),
|
||||
asset_id: Some(asset.asset_id.clone()),
|
||||
byte_len: Some(asset.byte_len),
|
||||
block_count: None,
|
||||
chunk_count: None,
|
||||
parser_version: None,
|
||||
chunker_version: None,
|
||||
warnings: vec![format!("unsupported media type: .{ext}")],
|
||||
error: None,
|
||||
});
|
||||
```
|
||||
|
||||
For the **kb:// URI** sites:
|
||||
|
||||
```rust
|
||||
warnings: vec!["kb:// URI not yet supported".to_string()],
|
||||
```
|
||||
|
||||
Add the helper near the other per-asset helpers:
|
||||
|
||||
```rust
|
||||
/// p9-fb-25: extract the lowercase extension (no leading dot) from a
|
||||
/// workspace path for use in the `unsupported media type: .X`
|
||||
/// warning + `IngestReport.skipped_by_extension` key. Returns
|
||||
/// `"<no-ext>"` for paths with no extension. Always lowercase so
|
||||
/// `Foo.DOCX` and `bar.docx` aggregate under the same key.
|
||||
fn ext_for_skip_warning(path: &str) -> String {
|
||||
std::path::Path::new(path)
|
||||
.extension()
|
||||
.and_then(|s| s.to_str())
|
||||
.map(|s| s.to_ascii_lowercase())
|
||||
.unwrap_or_else(|| "<no-ext>".to_string())
|
||||
}
|
||||
```
|
||||
|
||||
Note: for the `<no-ext>` case the warning should read `"unsupported media type: <no-ext>"` (no leading dot — sentinel). Adjust the format! call:
|
||||
|
||||
```rust
|
||||
let ext = ext_for_skip_warning(&asset.workspace_path.0);
|
||||
let warning = if ext == "<no-ext>" {
|
||||
"unsupported media type: <no-ext>".to_string()
|
||||
} else {
|
||||
format!("unsupported media type: .{ext}")
|
||||
};
|
||||
warnings: vec![warning],
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Bump `aggregate.skipped_by_extension` in the asset loop**
|
||||
|
||||
Find the asset loop in `ingest_with_config_opts` (search for `IngestItemKind::Skipped =>` arms in the per-result match). Where the loop currently does `aggregate.skipped += 1`, also bump the per-extension counter using the same `ext_for_skip_warning` helper:
|
||||
|
||||
```rust
|
||||
IngestItemKind::Skipped => {
|
||||
aggregate.skipped += 1;
|
||||
let ext = ext_for_skip_warning(&item.doc_path.0);
|
||||
*aggregate.skipped_by_extension.entry(ext).or_insert(0) += 1;
|
||||
}
|
||||
```
|
||||
|
||||
After the loop, when building the final `IngestReport`, populate `skipped_by_extension: aggregate.skipped_by_extension.clone()`.
|
||||
|
||||
- [ ] **Step 4: Run the test**
|
||||
|
||||
Run: `cargo test -p kebab-app --test skip_reason`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Run the full kebab-app suite for regressions**
|
||||
|
||||
Run: `cargo test -p kebab-app`
|
||||
Expected: existing tests pass. The change is additive — Skipped items previously had empty `warnings` and `skipped_by_extension` was 0. Tests that asserted `warnings.is_empty()` may need updating (search):
|
||||
|
||||
```bash
|
||||
grep -rn 'warnings.is_empty\|warnings, vec!\[\]\|warnings == Vec::new' crates/kebab-app/tests/
|
||||
```
|
||||
|
||||
Update any failing test to either skip the warnings assertion (if not the focus) or assert the new content.
|
||||
|
||||
- [ ] **Step 6: clippy + commit**
|
||||
|
||||
```bash
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
git add -u
|
||||
git commit -m "feat(kebab-app): p9-fb-25 task 5 — Skipped warnings + skipped_by_extension aggregation
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: CLI summary + TUI status_line render breakdown
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/kebab-cli/src/main.rs` (ingest summary print)
|
||||
- Modify: `crates/kebab-tui/src/ingest_progress.rs` (status_line)
|
||||
|
||||
- [ ] **Step 1: Update CLI summary**
|
||||
|
||||
Open `crates/kebab-cli/src/main.rs`. Find the human-mode summary print (search for `"new" =>` or similar around the ingest-finished branch). The current format is `"... {N} skipped, ..."`. Replace with:
|
||||
|
||||
```rust
|
||||
let skipped_breakdown = if report.skipped_by_extension.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
let mut entries: Vec<_> = report.skipped_by_extension.iter().collect();
|
||||
// desc by count, ties broken by key for stable output.
|
||||
entries.sort_by(|a, b| b.1.cmp(a.1).then_with(|| a.0.cmp(b.0)));
|
||||
let parts: Vec<String> = entries.iter().map(|(k, v)| format!("{v} {k}")).collect();
|
||||
format!(": {}", parts.join(", "))
|
||||
};
|
||||
println!(
|
||||
"✓ ingest: {} docs ({} new, {} updated, {} unchanged, {} skipped{}), {} chunks indexed in {}s",
|
||||
report.scanned,
|
||||
report.new,
|
||||
report.updated,
|
||||
report.unchanged,
|
||||
report.skipped,
|
||||
skipped_breakdown,
|
||||
/* chunks_indexed: derive from items or store in IngestReport */ 0, // adapt to actual
|
||||
report.duration_ms / 1000,
|
||||
);
|
||||
```
|
||||
|
||||
Adapt to the actual print site — the existing print may use `report.duration_ms` directly or have a `chunks_indexed` already plumbed. Match the existing surrounding pattern.
|
||||
|
||||
- [ ] **Step 2: Update TUI status_line**
|
||||
|
||||
Open `crates/kebab-tui/src/ingest_progress.rs`. Find the success branch in `pub fn status_line` (around line 170+). Apply the same breakdown logic:
|
||||
|
||||
```rust
|
||||
let skipped_breakdown = if state.counts.skipped_by_extension.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
let mut entries: Vec<_> = state.counts.skipped_by_extension.iter().collect();
|
||||
entries.sort_by(|a, b| b.1.cmp(a.1).then_with(|| a.0.cmp(b.0)));
|
||||
let parts: Vec<String> = entries.iter().map(|(k, v)| format!("{v} {k}")).collect();
|
||||
format!(": {}", parts.join(", "))
|
||||
};
|
||||
return format!(
|
||||
"✓ ingest: {} docs ({} new, {} updated, {} unchanged, {} skipped{}), {} chunks indexed in {}s",
|
||||
state.counts.scanned,
|
||||
state.counts.new,
|
||||
state.counts.updated,
|
||||
state.counts.unchanged,
|
||||
state.counts.skipped,
|
||||
skipped_breakdown,
|
||||
state.counts.chunks_indexed,
|
||||
secs,
|
||||
);
|
||||
```
|
||||
|
||||
Apply the same to the aborted branch:
|
||||
|
||||
```rust
|
||||
return format!(
|
||||
"✗ ingest aborted at {}/{} after {}s (new={} updated={} unchanged={} skipped={}{} errors={})",
|
||||
state.counts.scanned.saturating_sub(state.counts.errors),
|
||||
state.counts.scanned,
|
||||
secs,
|
||||
state.counts.new,
|
||||
state.counts.updated,
|
||||
state.counts.unchanged,
|
||||
state.counts.skipped,
|
||||
skipped_breakdown,
|
||||
state.counts.errors,
|
||||
);
|
||||
```
|
||||
|
||||
In-flight branch unchanged.
|
||||
|
||||
- [ ] **Step 3: Update existing status_line tests**
|
||||
|
||||
Find `#[cfg(test)] mod tests` in `crates/kebab-tui/src/ingest_progress.rs`. Existing tests construct `AggregateCounts` literals. After Task 4 they already have `skipped_by_extension: BTreeMap::new()`. For tests that exercise the breakdown, build an `AggregateCounts` with a populated map:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn status_line_includes_skipped_breakdown() {
|
||||
use std::collections::BTreeMap;
|
||||
let mut counts = AggregateCounts::default();
|
||||
counts.scanned = 10;
|
||||
counts.skipped = 3;
|
||||
counts.skipped_by_extension.insert("docx".into(), 2);
|
||||
counts.skipped_by_extension.insert("txt".into(), 1);
|
||||
let state = IngestState { /* fill mandatory fields */ };
|
||||
state.counts = counts;
|
||||
state.terminal_at = Some(std::time::Instant::now()); // make `if state.terminal_at.is_some()` true
|
||||
let line = status_line(&state);
|
||||
assert!(line.contains("3 skipped: 2 docx, 1 txt"), "got: {line}");
|
||||
}
|
||||
```
|
||||
|
||||
Adapt to the actual `IngestState` struct fields.
|
||||
|
||||
- [ ] **Step 4: Run + commit**
|
||||
|
||||
```bash
|
||||
cargo test -p kebab-cli -p kebab-tui --lib
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
git add -u
|
||||
git commit -m "feat(kebab-cli, kebab-tui): p9-fb-25 task 6 — render skipped-by-extension breakdown
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Docs sync
|
||||
|
||||
**Files:**
|
||||
- Modify: `README.md`, `HANDOFF.md`, `tasks/HOTFIXES.md`, `tasks/INDEX.md`
|
||||
- Create: `tasks/p9/p9-fb-25-config-include-removal.md`
|
||||
|
||||
- [ ] **Step 1: README**
|
||||
|
||||
Open `README.md`. Find the `kebab ingest` row (or paragraph if it's not a row). Append:
|
||||
|
||||
```
|
||||
**지원 형식** (extractor 자동 결정 — config 에 명시 불가): Markdown (`.md`), 이미지 (`.png` / `.jpg` / `.jpeg`, OCR + caption), PDF (`.pdf`). 다른 확장자는 자동 skip — `IngestItem.warnings` 에 사유 (`"unsupported media type: .docx"` 등), `IngestReport.skipped_by_extension` 에 카운트 분류, CLI / TUI summary 에 breakdown 표시.
|
||||
```
|
||||
|
||||
If the `kebab tui` row already mentions some of this, integrate into existing text instead of duplicating.
|
||||
|
||||
- [ ] **Step 2: HANDOFF.md**
|
||||
|
||||
Add a new entry directly above the most recent `p9-fb-24` row (or wherever the dated list begins):
|
||||
|
||||
```
|
||||
- **2026-05-05 P9 post-도그푸딩 (p9-fb-25)** — Config 의 `workspace.include` 필드 제거 + 지원 형식 가시성. 사용자 도그푸딩 피드백: include + exclude 동시 존재가 case 4 (둘 다 매치 안 함) 의미 모호 + 어차피 처리 가능 형식 (md / png / jpg / pdf) 이 정해져 있으니 명시 필요. `WorkspaceCfg.include` 제거 (옛 config 의 `include = [...]` 은 silently 무시 + 단발 deprecation warning). `IngestItem.warnings` 가 Skipped 시 사유 (`"unsupported media type: .docx"` 등) 채움. `IngestReport.skipped_by_extension: BTreeMap<String, u32>` 신규 (additive wire — release 트리거 안 됨). CLI / TUI summary 에 breakdown 표시 (`"5 skipped: 3 docx, 1 txt, 1 epub"`). README + `kebab init` 헤더 주석에 지원 형식 명시. spec: `tasks/p9/p9-fb-25-config-include-removal.md`. HOTFIXES `2026-05-05 — p9-fb-25` 가 source of truth.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: HOTFIXES.md**
|
||||
|
||||
Open `tasks/HOTFIXES.md`. Add new section above `## 2026-05-04 — p9-fb-24`:
|
||||
|
||||
```markdown
|
||||
## 2026-05-05 — p9-fb-25 (post-dogfooding): config workspace.include 제거 + 지원 형식 가시성
|
||||
|
||||
**Source feedback**: 사용자 도그푸딩 2026-05-05 — config 의 `workspace.include` + `workspace.exclude` 동시 존재가 case 4 (둘 다 매치 안 함) 의미 모호 + 어차피 처리 가능 형식 (md / png / jpg / pdf) 이 정해져 있으니 사용자에게 명시 필요.
|
||||
|
||||
**Live binding 변경**:
|
||||
|
||||
- `kebab-config::WorkspaceCfg.include: Vec<String>` 제거. denylist-only 모델. 옛 config 의 `include = [...]` 은 serde 가 silently 무시 + `Config::from_file` 가 단발 `tracing::warn!` 으로 deprecation 안내 (`std::sync::OnceLock` — 같은 process 안에서 한 번만).
|
||||
- `kebab-core::IngestItem.warnings` 가 Skipped 시 사유 채움: `"unsupported media type: .{ext}"` (ext 없으면 `"unsupported media type: <no-ext>"`) / `"kb:// URI not yet supported"`.
|
||||
- `kebab-core::IngestReport.skipped_by_extension: BTreeMap<String, u32>` + `kebab-app::AggregateCounts.skipped_by_extension` 신규. key = lowercase ext (`docx`, `txt`), no-ext sentinel = `<no-ext>`. wire schema `ingest_report.v1` 에 additive 추가 (v1 호환 유지 — release 트리거 안 됨 per CLAUDE.md release 규약).
|
||||
- CLI summary + TUI status_line final / aborted: `5 skipped: 3 docx, 1 txt, 1 epub` 형식. desc 정렬 + 모두 표시.
|
||||
- `kebab-app::init_workspace` 헤더 주석에 지원 형식 명시 (Markdown / 이미지 / PDF + 각 확장자).
|
||||
- README `kebab ingest` 설명에 지원 형식 + skip 사유 + breakdown 표시 명시.
|
||||
|
||||
**Spec contract impact**: design §6.2 의 `workspace.include` 항목 invalidate (frozen 그대로 두고 본 항목 + spec `tasks/p9/p9-fb-25-config-include-removal.md` 가 source of truth). design §3.x `IngestReport` + §2.4a `IngestEvent` 에 새 필드 / 새 warning 의미 추가 (additive).
|
||||
|
||||
**Tests added**: 약 6 신규 (kebab-config 단위 2: legacy include 무시 + WorkspaceCfg 필드 destructure / kebab-app 통합 1: skip_reason / kebab-tui 단위 1: breakdown 라인 / kebab-app 단위 1: init template 헤더 / kebab-app 단위 1: ext_for_skip_warning helper). 기존 723 워크스페이스 테스트 무수정 통과.
|
||||
|
||||
**Known limitation (deferred)**:
|
||||
|
||||
- `SourceScope.include` (`kebab-core::traits`) 는 그대로 — design §7.1 abstraction 이라 별 spec 으로 다룰 수 있음. 본 PR 은 config 단의 `WorkspaceCfg.include` 만 정리.
|
||||
- 새 extractor (txt / docx / epub 등) 도입은 별 spec.
|
||||
- `kebab doctor` 가 unsupported 파일 카운트 분석은 후속 task.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: INDEX.md**
|
||||
|
||||
Open `tasks/INDEX.md`. Append to the p9-fb section:
|
||||
|
||||
```
|
||||
- [p9-fb-25 config workspace.include 제거 + 지원 형식 가시성 (post-도그푸딩)](p9/p9-fb-25-config-include-removal.md)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Per-task spec file**
|
||||
|
||||
Create `tasks/p9/p9-fb-25-config-include-removal.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
phase: P9
|
||||
component: kebab-config
|
||||
task_id: p9-fb-25
|
||||
title: "Config workspace.include 제거 + 지원 형식 가시성 (post-merge dogfooding)"
|
||||
status: completed
|
||||
depends_on: [p9-fb-23]
|
||||
unblocks: []
|
||||
contract_source: ../../docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
|
||||
contract_sections: [§6.2 Workspace, §3.x IngestReport, §2.4a IngestEvent]
|
||||
source_feedback: 사용자 도그푸딩 2026-05-05 — include + exclude 의미 모호 + 지원 형식 가시성 부족.
|
||||
---
|
||||
|
||||
# p9-fb-25 — Config `workspace.include` 제거 + 지원 형식 가시성
|
||||
|
||||
상세 설계: `docs/superpowers/specs/2026-05-05-p9-fb-25-config-include-removal-design.md`.
|
||||
구현 계획: `docs/superpowers/plans/2026-05-05-p9-fb-25-config-include-removal.md`.
|
||||
|
||||
## Goal
|
||||
|
||||
- `WorkspaceCfg.include` 필드 제거 (denylist-only 모델 정착).
|
||||
- 사용자가 ingest 결과에서 어떤 파일이 왜 skip 됐는지 즉시 파악.
|
||||
- 지원 형식 (md / png / jpg / pdf) 을 README + `kebab init` config 주석에 명시.
|
||||
|
||||
## Behavior contract
|
||||
|
||||
- 옛 config 의 `include = [...]` 은 silently 무시 + 단발 deprecation warning.
|
||||
- Skipped 시 `IngestItem.warnings` = `["unsupported media type: .ext"]` 또는 `["unsupported media type: <no-ext>"]` 또는 `["kb:// URI not yet supported"]`.
|
||||
- `IngestReport.skipped_by_extension` = `BTreeMap<lowercase-ext, count>`. no-ext 키 = `<no-ext>`.
|
||||
- CLI / TUI summary final / aborted 라인에 `"N skipped: A docx, B txt, ..."` (desc 정렬, 모두).
|
||||
|
||||
## Tests
|
||||
|
||||
- legacy include 무시 + 새 WorkspaceCfg 필드 destructure (kebab-config).
|
||||
- skip_reason 통합 (kebab-app): docx + Makefile 두 파일 ingest → warnings + skipped_by_extension 채워짐.
|
||||
- status_line breakdown (kebab-tui).
|
||||
- init template 헤더 (kebab-app).
|
||||
- ext_for_skip_warning helper (kebab-app).
|
||||
|
||||
## Risks / notes
|
||||
|
||||
- 옛 config 가 narrow allowlist (예: `include = ["**/*.md"]`) 면 본 변경 후 `.png` 등이 자동 ingest 시작 — deprecation warning + README 가 alarm.
|
||||
- `SourceScope.include` (kebab-core) 는 그대로.
|
||||
|
||||
Live deviations 반영 위치: `tasks/HOTFIXES.md` `2026-05-05 — p9-fb-25` 항목.
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Final commit**
|
||||
|
||||
```bash
|
||||
git add README.md HANDOFF.md tasks/HOTFIXES.md tasks/INDEX.md tasks/p9/p9-fb-25-config-include-removal.md
|
||||
git commit -m "docs(p9-fb-25): README + HANDOFF + HOTFIXES + INDEX + per-task spec
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Notes (writer)
|
||||
|
||||
**Spec coverage:**
|
||||
- `WorkspaceCfg.include` 제거 + deprecation warning → Task 1.
|
||||
- SourceScope construction cleanup → Task 2.
|
||||
- `kebab init` header supported-extensions → Task 3.
|
||||
- `IngestReport.skipped_by_extension` + AggregateCounts + wire schema → Task 4.
|
||||
- `IngestItem.warnings` populate + asset loop bumps → Task 5.
|
||||
- CLI / TUI summary breakdown render → Task 6.
|
||||
- README + docs sync → Task 7.
|
||||
|
||||
**Type / API consistency:**
|
||||
- `BTreeMap<String, u32>` used in both `IngestReport.skipped_by_extension` (Task 4) and `AggregateCounts.skipped_by_extension` (Task 4) — same type.
|
||||
- `<no-ext>` sentinel used in BOTH `IngestItem.warnings` (Task 5) and `BTreeMap` key (Task 5).
|
||||
- `ext_for_skip_warning` helper defined in Task 5, consumed by Tasks 5 + 6 (CLI + TUI consume the resulting `BTreeMap`, not the helper directly).
|
||||
- `Copy` derive removed from `AggregateCounts` (Task 4) — callers using `let counts = state.counts;` continue to compile because `Clone` still works (assignment of non-Copy type via move; Rust borrow checker handles).
|
||||
|
||||
**Placeholder scan:** Each step has full code. Adapter-language ("adapt to actual existing helper") is reserved for genuine ambiguity (CLI summary print site, init template test fixture pattern) — the engineer must inspect 5-10 lines of context.
|
||||
|
||||
**Risks documented:**
|
||||
- `Copy` removal on `AggregateCounts` may surface compile errors at call sites that rely on `Copy`. Plan flags this in Task 4 step 2 with grep instruction.
|
||||
- Deprecation warning might fire from the `kebab init` test if it produces a config with `include = [...]` first. Task 3's test uses `force=true` on a fresh dir → no `include` in default → no warning. Acceptable.
|
||||
- `set_var(XDG_CONFIG_HOME)` in init test relies on Rust 2024 `unsafe`. Plan flags the wrapping requirement.
|
||||
@@ -0,0 +1,166 @@
|
||||
# p9-fb-25 — Config `workspace.include` 제거 + 지원 형식 가시성
|
||||
|
||||
**Date**: 2026-05-05
|
||||
**Status**: planned
|
||||
**Audience**: kebab-config / kebab-app / kebab-cli / kebab-tui implementer.
|
||||
**Source feedback**: 사용자 도그푸딩 2026-05-05 — config 의 `workspace.include` + `workspace.exclude` 가 동시에 있으면 case 4 (둘 다 매치 안 함) 의미 모호 + 어차피 처리 가능 형식이 정해져 있으니 사용자에게 명시 필요.
|
||||
|
||||
## Goal
|
||||
|
||||
- `WorkspaceCfg.include` 필드 제거. dead config field 제거 + denylist-only 모델 정착.
|
||||
- 사용자가 ingest 결과에서 \*\*어떤 파일이 왜 skip 됐는지\*\* 즉시 파악.
|
||||
- 지원 형식 (md / png / jpg / pdf) 을 README + `kebab init` config 주석에 명시.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- include 의 enforce 로직 추가 (반대 방향).
|
||||
- 새 extractor (txt / docx / epub 등) 도입 — 별 spec.
|
||||
- `kebab doctor` 가 unsupported 파일 카운트 분석 — 별 task (간단 follow-up 가능).
|
||||
|
||||
## Allowed dependencies
|
||||
|
||||
- 기존 crate 만. 신규 crate 없음. 신규 SQLite migration 없음.
|
||||
|
||||
## Storage 변경
|
||||
|
||||
없음.
|
||||
|
||||
## API / Wire 변경
|
||||
|
||||
### `kebab-config::WorkspaceCfg`
|
||||
|
||||
`include: Vec<String>` 필드 제거. `exclude: Vec<String>` 만 유지.
|
||||
|
||||
backward-compat: serde default `deny_unknown_fields` 미사용이라 옛 config 의 `include = [...]` 은 silently deserialize 통과 + 무시. `Config::load` 가 옛 키 발견 시 `tracing::warn!` 로 deprecation 경고 emit (단발 — 같은 process 안에서 한 번만):
|
||||
|
||||
```
|
||||
deprecated config: `workspace.include` 필드는 더 이상 사용되지 않습니다 (p9-fb-25). 처리 가능한 형식 (md / png / jpg / pdf) 은 extractor 가 자동 결정. 다음 버전부터 config 갱신 권장.
|
||||
```
|
||||
|
||||
검출 방법: `Config::load` 가 raw TOML 파싱 후 `workspace` 테이블의 키 이름을 살펴 `include` 존재 여부 확인. `serde_ignored` crate 미도입 (YAGNI) — `toml::Value` 로 raw lookup 한 번.
|
||||
|
||||
### `kebab-core::IngestItem.warnings`
|
||||
|
||||
Skipped path 가 빈 `Vec` 대신 사유 한 줄 채움. 두 case:
|
||||
|
||||
- media-type filter (extractor 미지원): `format!("unsupported media type: .{ext}")` (e.g. `"unsupported media type: .docx"`). extension 이 없으면 `"unsupported media type: <no-ext>"`.
|
||||
- `kb://` URI: `"kb:// URI not yet supported"`.
|
||||
|
||||
### `kebab-core::IngestReport.skipped_by_extension`
|
||||
|
||||
신규 필드:
|
||||
|
||||
```rust
|
||||
pub skipped_by_extension: std::collections::BTreeMap<String, u32>,
|
||||
```
|
||||
|
||||
key = lowercase extension without leading dot (`"docx"`, `"txt"`, `"epub"`). 확장자 없는 파일 = `"<no-ext>"` sentinel (꺾쇠로 일반 ext 와 시각 구분).
|
||||
|
||||
`BTreeMap` 사용 — wire JSON 안에서 key 정렬 안정. `HashMap` 은 매 직렬화마다 순서 바뀌어 diff / snapshot 테스트 noisy.
|
||||
|
||||
`AggregateCounts` 도 동일 필드 추가 — TUI / CLI 가 in-flight 와 final 모두에서 일관 표시.
|
||||
|
||||
### Wire schema `ingest_report.v1`
|
||||
|
||||
`skipped_by_extension` 필드 additive 추가:
|
||||
|
||||
```json
|
||||
"skipped_by_extension": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"description": "p9-fb-25: per-extension skip count. Key = lowercase extension without leading dot (e.g. 'docx'). Files without extension key under '<no-ext>'."
|
||||
}
|
||||
```
|
||||
|
||||
CLAUDE.md 의 release 규약 (additive minor) 에 따라 release bump 트리거 안 됨.
|
||||
|
||||
## TUI / CLI 노출
|
||||
|
||||
### CLI summary
|
||||
|
||||
기존:
|
||||
|
||||
```
|
||||
✓ ingest: 100 docs (5 new, 3 updated, 2 unchanged, 90 skipped), 142 chunks indexed in 12s
|
||||
```
|
||||
|
||||
변경 (skipped > 0 + breakdown 있을 때만 괄호 안):
|
||||
|
||||
```
|
||||
✓ ingest: 100 docs (5 new, 3 updated, 2 unchanged, 90 skipped: 80 docx, 5 txt, 5 epub), 142 chunks indexed in 12s
|
||||
```
|
||||
|
||||
extension 카운트 desc 정렬 (큰 거 먼저). 모두 표시 (top-3 제한 없음). Line 길어질 우려가 있으나 사용자 원함 — line wrap 은 terminal 책임.
|
||||
|
||||
### TUI
|
||||
|
||||
`kebab-tui::ingest_progress::status_line` 의 final / aborted 라인 동일 포맷. in-flight 진행 중에는 breakdown 표시 안 함 (idx 진행 중 계속 변동, 불필요 noise).
|
||||
|
||||
## 사용자 안내 (docs)
|
||||
|
||||
### README
|
||||
|
||||
`kebab ingest` row 의 cell 끝에 추가:
|
||||
|
||||
```
|
||||
**지원 형식** (extractor 자동 결정): Markdown (`.md`) / 이미지 (`.png`, `.jpg`, `.jpeg`, OCR + caption) / PDF (`.pdf`). 다른 확장자는 자동 skip — `--json` / TUI 의 `IngestItem.warnings` 에 사유 (`unsupported media type: .docx` 등). 카운트 분류는 `IngestReport.skipped_by_extension`.
|
||||
```
|
||||
|
||||
### `kebab init` config.toml 주석
|
||||
|
||||
`[workspace]` section 위에 주석 추가:
|
||||
|
||||
```toml
|
||||
# [workspace] — 색인 대상 디렉토리 + denylist.
|
||||
#
|
||||
# 지원 형식 (extractor 가 자동 결정 — config 에 명시할 수 없음):
|
||||
# - Markdown: .md
|
||||
# - 이미지: .png .jpg .jpeg (OCR + caption)
|
||||
# - PDF: .pdf
|
||||
#
|
||||
# 다른 확장자는 ingest 시 자동 skip + warning. 처리 대상 폴더의
|
||||
# 일부만 ingest 하고 싶으면 `kebab ingest <path>` 로 root 명시
|
||||
# 또는 `.kebabignore` 파일 / 본 `exclude` 로 denylist.
|
||||
[workspace]
|
||||
root = "..."
|
||||
exclude = [...]
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
### 신규 단위
|
||||
|
||||
- `kebab-config`: `Config::load` 가 옛 `include = [...]` 발견 시 warning emit + 정상 deserialize. snapshot test (in-memory string TOML).
|
||||
- `kebab-core`: `IngestItem` JSON serde — `warnings` 가 `["unsupported media type: .docx"]` round-trip.
|
||||
- `kebab-core`: `IngestReport.skipped_by_extension` JSON serde — `BTreeMap` 정렬 stable.
|
||||
|
||||
### 신규 통합
|
||||
|
||||
- `kebab-app`: 다양한 확장자 mix (`.md`, `.docx`, `.txt`, no-ext 파일) workspace 에서 ingest → `report.skipped_by_extension == {"docx": 1, "txt": 1, "<no-ext>": 1}` + 각 skipped 의 `warnings` 채워짐.
|
||||
- `kebab-tui`: `status_line` 가 `90 skipped: 80 docx, 5 txt, 5 epub` 형식.
|
||||
- `kebab-cli`: `kebab ingest --json` 출력에 `skipped_by_extension` 필드.
|
||||
|
||||
### 기존 영향
|
||||
|
||||
- 기존 `IngestReport` 구성 site (테스트 fixture 등) 가 새 필드 default 로 채움 (`BTreeMap::new()`).
|
||||
- `WorkspaceCfg` 의 `include` 필드 제거로 컴파일 에러 → 매 site 정리 (기존 default 가 `vec!["**/*.md"]` 였으니 모두 제거).
|
||||
|
||||
## Spec contract impact
|
||||
|
||||
- design §6.2 의 `workspace.include` 항목 invalidate. frozen spec 그대로 두고 본 spec + HOTFIXES `2026-05-05 — p9-fb-25` 가 source of truth.
|
||||
- design §3.x `IngestReport` 에 `skipped_by_extension` 필드 추가 (additive).
|
||||
- design §2.4a `IngestEvent::AssetFinished` 에 새로 emit 되는 warnings 의미 추가 (variant 변경 없음, content 풍부화).
|
||||
|
||||
## Risks / notes
|
||||
|
||||
- **옛 config 가 `include = ["**/*.md"]` 같은 narrow 한 allowlist** 면 본 변경 후 그 이상의 확장자 (예: 파일 추가된 `.png`) 가 자동 ingest 시작. 사용자 의도와 어긋날 수 있음. 완화: deprecation warning 의 문구가 \"처리 가능 형식 자동 결정\" 명시 → 사용자가 alarm 받음. + README 변경. 경계 case 라 design accepted.
|
||||
- **`skipped_by_extension` 용량**: workspace 가 1만 파일이면 dict size 작음 (extension 종류는 보통 < 50). wire 영향 무시.
|
||||
- **deprecation warning 단발 vs every-load**: `Config::load` 가 매 CLI 호출마다 발생. 단발 (`std::sync::Once`) 이 깔끔. 본 spec 은 단발 채택.
|
||||
- **release 트리거**: wire schema additive + serde backward-compat → CLAUDE.md release 규약 의 minor 트리거에 해당 안 됨 (additive 만으로 release 안 찍음). 사용자 explicit 도그푸딩 요청 시 bump.
|
||||
|
||||
## Live deviations
|
||||
|
||||
추후 발견되는 deviation 은 `tasks/HOTFIXES.md` `2026-05-05 — p9-fb-25` 항목에 기록.
|
||||
@@ -13,7 +13,8 @@
|
||||
"skipped",
|
||||
"unchanged",
|
||||
"errors",
|
||||
"duration_ms"
|
||||
"duration_ms",
|
||||
"skipped_by_extension"
|
||||
],
|
||||
"properties": {
|
||||
"schema_version": { "const": "ingest_report.v1" },
|
||||
@@ -29,6 +30,14 @@
|
||||
},
|
||||
"errors": { "type": "integer", "minimum": 0 },
|
||||
"duration_ms": { "type": "integer", "minimum": 0 },
|
||||
"skipped_by_extension": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"description": "p9-fb-25: per-extension skip count. Key = lowercase extension without leading dot (e.g. 'docx'). Files without extension key under '<no-ext>'."
|
||||
},
|
||||
"items": { "type": ["array", "null"] }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,6 +14,29 @@ historical contract that was implemented; this file accumulates the
|
||||
deltas so phase 5+ readers can find the live behavior without diffing
|
||||
git history.
|
||||
|
||||
## 2026-05-05 — p9-fb-25 (post-dogfooding): config workspace.include 제거 + 지원 형식 가시성
|
||||
|
||||
**Source feedback**: 사용자 도그푸딩 2026-05-05 — config 의 `workspace.include` + `workspace.exclude` 동시 존재가 case 4 (둘 다 매치 안 함) 의미 모호 + 어차피 처리 가능 형식 (md / png / jpg / pdf) 이 정해져 있으니 사용자에게 명시 필요.
|
||||
|
||||
**Live binding 변경**:
|
||||
|
||||
- `kebab-config::WorkspaceCfg.include: Vec<String>` 제거. denylist-only 모델. 옛 config 의 `include = [...]` 은 serde 가 silently 무시 + `Config::from_file` 가 단발 `tracing::warn!` 으로 deprecation 안내 (`std::sync::OnceLock` — 같은 process 안에서 한 번만).
|
||||
- `kebab-core::IngestItem.warnings` 가 Skipped 시 사유 채움: `"unsupported media type: .{ext}"` (ext 없으면 `"unsupported media type: <no-ext>"`) / `"kb:// URI not yet supported"`.
|
||||
- `kebab-core::IngestReport.skipped_by_extension: BTreeMap<String, u32>` + `kebab-app::AggregateCounts.skipped_by_extension` 신규. key = lowercase ext (`docx`, `txt`), no-ext sentinel = `<no-ext>`. wire schema `ingest_report.v1` 에 additive 추가 (v1 호환 유지 — release 트리거 안 됨 per CLAUDE.md release 규약).
|
||||
- CLI summary + TUI status_line final / aborted: `5 skipped: 3 docx, 1 txt, 1 epub` 형식. desc 정렬 (count) + ties by key alphabetic + 모두 표시.
|
||||
- `kebab-app::init_workspace` 헤더 주석에 지원 형식 명시 (Markdown / 이미지 / PDF + 각 확장자).
|
||||
- README `kebab ingest` 설명에 지원 형식 + skip 사유 + breakdown 표시 명시.
|
||||
|
||||
**Spec contract impact**: design §6.2 의 `workspace.include` 항목 invalidate (frozen 그대로 두고 본 항목 + spec `tasks/p9/p9-fb-25-config-include-removal.md` 가 source of truth). design §3.x `IngestReport` + §2.4a `IngestEvent` 에 새 필드 / 새 warning 의미 추가 (additive).
|
||||
|
||||
**Tests added**: 5 신규 (kebab-config 단위 2: legacy include 무시 + WorkspaceCfg 필드 destructure / kebab-app 통합 1: skip_reason / kebab-app 통합 1: init_template 헤더 / kebab-tui 단위 2: status_line breakdown 완료/abort) + 1 unit (kebab-app 의 render_skipped_breakdown). 기존 fixture 6 개 mechanical adapter 수정 (`tests/common/mod.rs` SourceScope, `tests/image_pipeline.rs` × 2 + `tests/pdf_pipeline.rs` 의 dead `include.push` 제거, `tests/ingest_report_snapshot.rs` + `kebab-cli/src/wire.rs` literal 에 `BTreeMap::new()` 추가, snapshot JSON 의 `skipped_by_extension` 필드). assertion 의미 변경 없음.
|
||||
|
||||
**Known limitation (deferred)**:
|
||||
|
||||
- `SourceScope.include` (`kebab-core::traits`) 는 그대로 — design §7.1 abstraction 이라 별 spec 으로 다룰 수 있음. 본 PR 은 config 단의 `WorkspaceCfg.include` 만 정리.
|
||||
- 새 extractor (txt / docx / epub 등) 도입은 별 spec.
|
||||
- `kebab doctor` 가 unsupported 파일 카운트 분석은 후속 task.
|
||||
|
||||
## 2026-05-04 — p9-fb-23 (post-dogfooding): Incremental ingest
|
||||
|
||||
**Source feedback**: 사용자 도그푸딩 2026-05-04 — "새 문서들이 폴더에 추가되면 ingest 시 변하지 않은 문서는 다시 ingest 하지 않고 변하거나 새로 추가된 문서만 처리하고 싶어."
|
||||
|
||||
@@ -108,6 +108,7 @@ P0~P5 는 직렬. P6~P9 는 P5 이후 병렬 가능.
|
||||
- [p9-fb-22 cursor mid-string editing + Ask follow-tail (post-도그푸딩)](p9/p9-fb-22-tui-cursor-and-autoscroll.md)
|
||||
- [p9-fb-23 incremental ingest (post-도그푸딩)](p9/p9-fb-23-incremental-ingest.md)
|
||||
- [p9-fb-24 status bar + Library header + page scroll (post-도그푸딩)](p9/p9-fb-24-tui-affordances.md)
|
||||
- [p9-fb-25 config workspace.include 제거 + 지원 형식 가시성 (post-도그푸딩)](p9/p9-fb-25-config-include-removal.md)
|
||||
|
||||
## Post-merge 핫픽스
|
||||
|
||||
|
||||
44
tasks/p9/p9-fb-25-config-include-removal.md
Normal file
44
tasks/p9/p9-fb-25-config-include-removal.md
Normal file
@@ -0,0 +1,44 @@
|
||||
---
|
||||
phase: P9
|
||||
component: kebab-config
|
||||
task_id: p9-fb-25
|
||||
title: "Config workspace.include 제거 + 지원 형식 가시성 (post-merge dogfooding)"
|
||||
status: completed
|
||||
depends_on: [p9-fb-23]
|
||||
unblocks: []
|
||||
contract_source: ../../docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
|
||||
contract_sections: [§6.2 Workspace, §3.x IngestReport, §2.4a IngestEvent]
|
||||
source_feedback: 사용자 도그푸딩 2026-05-05 — include + exclude 의미 모호 + 지원 형식 가시성 부족.
|
||||
---
|
||||
|
||||
# p9-fb-25 — Config `workspace.include` 제거 + 지원 형식 가시성
|
||||
|
||||
상세 설계: `docs/superpowers/specs/2026-05-05-p9-fb-25-config-include-removal-design.md`.
|
||||
구현 계획: `docs/superpowers/plans/2026-05-05-p9-fb-25-config-include-removal.md`.
|
||||
|
||||
## Goal
|
||||
|
||||
- `WorkspaceCfg.include` 필드 제거 (denylist-only 모델 정착).
|
||||
- 사용자가 ingest 결과에서 어떤 파일이 왜 skip 됐는지 즉시 파악.
|
||||
- 지원 형식 (md / png / jpg / pdf) 을 README + `kebab init` config 주석에 명시.
|
||||
|
||||
## Behavior contract
|
||||
|
||||
- 옛 config 의 `include = [...]` 은 silently 무시 + 단발 deprecation warning.
|
||||
- Skipped 시 `IngestItem.warnings` = `["unsupported media type: .ext"]` 또는 `["unsupported media type: <no-ext>"]` 또는 `["kb:// URI not yet supported"]`.
|
||||
- `IngestReport.skipped_by_extension` = `BTreeMap<lowercase-ext, count>`. no-ext 키 = `<no-ext>`.
|
||||
- CLI / TUI summary final / aborted 라인에 `"N skipped: A docx, B txt, ..."` (desc 정렬, 모두 표시, ties by key alphabetic).
|
||||
|
||||
## Tests
|
||||
|
||||
- legacy include 무시 + 새 WorkspaceCfg 필드 destructure (kebab-config).
|
||||
- skip_reason 통합 (kebab-app): docx + Makefile 두 파일 ingest → warnings + skipped_by_extension 채워짐.
|
||||
- init_template 헤더 (kebab-app).
|
||||
- status_line breakdown 완료 / abort (kebab-tui).
|
||||
|
||||
## Risks / notes
|
||||
|
||||
- 옛 config 가 narrow allowlist (예: `include = ["**/*.md"]`) 면 본 변경 후 `.png` 등이 자동 ingest 시작 — deprecation warning + README 가 alarm.
|
||||
- `SourceScope.include` (kebab-core) 는 그대로.
|
||||
|
||||
Live deviations 반영 위치: `tasks/HOTFIXES.md` `2026-05-05 — p9-fb-25` 항목.
|
||||
Reference in New Issue
Block a user