kebab/docs/superpowers/plans/2026-05-10-p9-fb-39b-embedding-upgrade.md

fb-39b Embedding Model Upgrade Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Upgrade default embedding model from multilingual-e5-small (384 dim) to multilingual-e5-large (1024 dim) so retrieval precision can improve on Korean dogfooding corpus. Existing user TOMLs pinning multilingual-e5-small keep working unchanged.

Architecture: Three-line code surface: a new arm in kebab-embed-local::resolve_model, defaults flipped in kebab-config::Config::defaults (and the TOML template), and the existing test asserting the 384 default updated. LanceDB tables are already namespaced by (model, dim) so an upgraded model writes to a fresh table; fb-23 incremental ingest detects the embedding_version mismatch and auto-re-embeds on next ingest. No migration tooling — orphan old-model tables cleaned via kebab reset --vector-only.

Tech Stack: Rust 2024, fastembed 4.9.1 (MultilingualE5Large enum already shipped), LanceDB.

Spec: docs/superpowers/specs/2026-05-10-p9-fb-39b-embedding-upgrade-design.md

---

File map

Modify:

• crates/kebab-embed-local/src/lib.rs — add multilingual-e5-large arm in resolve_model. Update or add check_dim test for 1024.
• crates/kebab-config/src/lib.rs — flip Config::defaults().models.embedding.{model, dimensions} and the TOML template at line ~952. Update default test at line 767.
• README.md — [models.embedding] section: mention new default + small opt-out + dim mismatch hint.
• docs/SMOKE.md — append "Embedding upgrade (fb-39b)" walkthrough showing the kebab reset --vector-only && kebab ingest sequence + first-run ONNX download warning.
• docs/superpowers/specs/2026-04-27-kebab-final-form-design.md §5 storage / §9 versioning — update default model + dim references.
• tasks/HOTFIXES.md — entry for embedding upgrade UX (orphan tables on model swap, reset --vector-only flow).
• tasks/p9/p9-fb-39-retrieval-precision-tuning.md banner — append note "fb-39b lever 적용 (embedding upgrade) ✅".
• tasks/INDEX.md — fb-39b row ✅ (new row alongside fb-39).

Create:

• tasks/p9/p9-fb-39b-embedding-upgrade.md — new task spec mirroring fb-39 frontmatter (status: completed, design + plan links).

---

Task 1: Add multilingual-e5-large to kebab-embed-local

Files:

• Modify: crates/kebab-embed-local/src/lib.rs

• Step 1: Append failing tests

Find the existing mod tests (~line 230). Append:

#[test]
fn resolve_model_supports_e5_large() {
    let m = resolve_model("multilingual-e5-large").expect("e5-large should resolve");
    // The fastembed enum is non-comparable in some versions; we only need
    // to confirm Ok and that the underlying TextEmbedding could be built.
    // Avoid actually constructing the model in tests (1.3 GB ONNX download).
    let _ = m;
}

#[test]
fn check_dim_passes_for_1024() {
    check_dim(1024, 1024).expect("matching dims must pass");
}

#[test]
fn check_dim_rejects_384_vs_1024() {
    let err = check_dim(384, 1024).expect_err("dim mismatch must error");
    let msg = format!("{err}");
    assert!(msg.contains("384") && msg.contains("1024"),
        "error must mention both dims, got: {msg}");
}

• Step 2: Run tests to confirm failures

cargo test -p kebab-embed-local resolve_model_supports_e5_large
cargo test -p kebab-embed-local check_dim_passes_for_1024

Expected: resolve_model_supports_e5_large fails (no arm); check_dim_* passes already (helper is generic).

• Step 3: Add arm to resolve_model

In crates/kebab-embed-local/src/lib.rs, find fn resolve_model (~line 199). Replace the match body:

fn resolve_model(name: &str) -> Result<EmbeddingModel> {
    match name {
        "multilingual-e5-small" => Ok(EmbeddingModel::MultilingualE5Small),
        "multilingual-e5-large" => Ok(EmbeddingModel::MultilingualE5Large),
        other => anyhow::bail!(
            "kb-embed-local: unsupported embedding model {other:?}; \
             this adapter currently ships `multilingual-e5-small` and \
             `multilingual-e5-large`. Add a new arm to `resolve_model` \
             (and a fastembed feature flag if needed) to support more."
        ),
    }
}

• Step 4: Run tests — all pass

cargo test -p kebab-embed-local
cargo clippy -p kebab-embed-local --all-targets -- -D warnings

• Step 5: Commit

git add crates/kebab-embed-local/src/lib.rs
git commit -m "feat(embed): add multilingual-e5-large arm to resolve_model (fb-39b)"

---

Task 2: Flip kebab-config default to e5-large + 1024 dim

Files:

• Modify: crates/kebab-config/src/lib.rs

• Step 1: Read existing default test + value sites

grep -n "multilingual-e5-small\|dimensions: 384\|dimensions = 384\|default.*embedding" crates/kebab-config/src/lib.rs

Three sites to update:

• Config::defaults() body (~line 307): dimensions: 384 and model: "multilingual-e5-small".

• Default-assert test (~line 767): assert_eq!(c.models.embedding.dimensions, 384) and likely a sibling assertion on model.

• TOML template at ~line 952: dimensions = 384 (and likely model = "multilingual-e5-small").

• Step 2: Add failing assertion to existing default test

Find the test at ~line 763-768 (likely defaults_match_design_64_score_gate or similar). Read it:

sed -n '760,780p' crates/kebab-config/src/lib.rs

If the test asserts dimensions == 384, change to 1024. If it doesn't assert model name, add:

    assert_eq!(c.models.embedding.model, "multilingual-e5-large");
    assert_eq!(c.models.embedding.dimensions, 1024);

• Step 3: Run tests — expect failure

cargo test -p kebab-config defaults_match

Expected: assertion failure on dimensions == 1024 (still 384) and/or model name.

• Step 4: Flip the defaults

In crates/kebab-config/src/lib.rs:307 (the EmbeddingCfg defaults block):

EmbeddingCfg {
    provider: "fastembed".to_string(),
    model: "multilingual-e5-large".to_string(),
    version: "v1".to_string(),
    dimensions: 1024,
    // ... preserve other fields (batch_size etc.) ...
}

(Read the surrounding lines first to confirm field names — if version field doesn't exist or has a different shape, only update model + dimensions.)

• Step 5: Flip the TOML template

In crates/kebab-config/src/lib.rs near line 952, the multi-line raw string contains the example TOML config. Find:

[models.embedding]
provider = "fastembed"
model = "multilingual-e5-small"
...
dimensions = 384

Replace with model = "multilingual-e5-large" and dimensions = 1024.

• Step 6: Run tests — pass

cargo test -p kebab-config
cargo clippy -p kebab-config --all-targets -- -D warnings

• Step 7: Commit

git add crates/kebab-config/src/lib.rs
git commit -m "feat(config): default embedding model multilingual-e5-large + 1024 dim (fb-39b)"

---

Task 3: Cross-crate test fixture sweep

Files:

• Modify: any test fixture broken by Task 2's default flip.

• Step 1: Find broken sites

cargo build --workspace 2>&1 | tail -10
cargo test --workspace --no-run 2>&1 | grep -E "error\[|FAILED" | head -20

Likely candidates:

• crates/kebab-app/tests/ — anywhere a test asserted embedding.dimensions == 384.
• crates/kebab-cli/tests/cli_schema.rs — a capability/model assertion may include the embedding model name.

For each failure, decide:

• Pin to small intentionally (test exercises small-specific behavior): set cfg.models.embedding.model = "multilingual-e5-small"; cfg.models.embedding.dimensions = 384; explicitly.
• Inherit new default (test just snapshots defaults): update assertion to multilingual-e5-large / 1024.

The vast majority of integration tests use provider = "none" (no embeddings) — those are unaffected.

• Step 2: Verify workspace builds

cargo build --workspace 2>&1 | tail -5

• Step 3: Run workspace tests

cargo test --workspace --no-fail-fast -j 1 2>&1 | tail -10
cargo clippy --workspace --all-targets -- -D warnings 2>&1 | tail -5

-j 1 REQUIRED.

Expected: all green.

• Step 4: Commit

git add crates/
git commit -m "fix(fb-39b): update test fixtures for embedding default flip"

(Skip this commit if cargo build --workspace is already clean after Task 2 — meaning no fixture broke.)

---

Task 4: Wire schema docs (design + HOTFIXES + new task spec)

Files:

• Modify: docs/superpowers/specs/2026-04-27-kebab-final-form-design.md

• Modify: tasks/HOTFIXES.md

• Create: tasks/p9/p9-fb-39b-embedding-upgrade.md

• Modify: tasks/p9/p9-fb-39-retrieval-precision-tuning.md

• Modify: tasks/INDEX.md

• Step 1: Update design §5 storage and §9 versioning

grep -n "multilingual-e5-small\|^## §5\|^### §5\|^## §9\|384" docs/superpowers/specs/2026-04-27-kebab-final-form-design.md | head -10

Update any reference to multilingual-e5-small or dim 384 in the design doc to read multilingual-e5-large and dim 1024. Keep historical version mentions intact (e.g. "0.6.0 shipped with multilingual-e5-small") if any — but the "current default" line must reflect the new model.

• Step 2: Add HOTFIXES entry

Append to tasks/HOTFIXES.md (under the dated log; place at top of the dated entries with today's date 2026-05-10):

- **2026-05-10 fb-39b — embedding upgrade UX**: default embedding flipped from `multilingual-e5-small` (384 dim) to `multilingual-e5-large` (1024 dim). LanceDB tables are namespaced by `(model, dim)` so the new model writes to a fresh table and the old `chunk_embeddings_multilingual-e5-small_384` table becomes orphan. fb-23 incremental ingest auto-re-embeds chunks (embedding_version mismatch) into the new table on next `kebab ingest`. To free disk before re-ingest, run `kebab reset --vector-only` first — this wipes both LanceDB and the SQLite `embedding_records` table. Search/ask against the new model returns empty hits until `kebab ingest` populates the new table.

• Step 3: Create tasks/p9/p9-fb-39b-embedding-upgrade.md

Mirror the fb-39 frontmatter shape:

---
phase: P9
component: kebab-embed-local + kebab-config + kebab-store-vector + docs
task_id: p9-fb-39b
title: "Embedding model upgrade (multilingual-e5-large)"
status: completed
target_version: 0.7.0
depends_on: [p9-fb-39]
unblocks: []
contract_source: ../../docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
contract_sections: [§4 search, §5 storage, §9 versioning cascade]
source_feedback: 사용자 도그푸딩 2026-05-06 — Claude Code 가 kebab CLI 사용 후 "rank 5+ 노이즈 섞임" 지적 (fb-39 의 lever 적용 측면).
---

# p9-fb-39b — Embedding model upgrade

> ✅ **구현 완료.** fb-39 의 lever 후보 4개 중 embedding model 업그레이드 lever 적용. P@k metric (fb-39) 으로 small vs large 비교 가능.
>
> - Design: [`docs/superpowers/specs/2026-05-10-p9-fb-39b-embedding-upgrade-design.md`](../../docs/superpowers/specs/2026-05-10-p9-fb-39b-embedding-upgrade-design.md)
> - Plan: [`docs/superpowers/plans/2026-05-10-p9-fb-39b-embedding-upgrade.md`](../../docs/superpowers/plans/2026-05-10-p9-fb-39b-embedding-upgrade.md)

## 요약

- `multilingual-e5-small` (384 dim) → `multilingual-e5-large` (1024 dim) default flip.
- 기존 user TOML 이 small 명시 시 그대로 (backwards-compat).
- fb-23 incremental ingest 가 embedding_version mismatch 감지 → 자동 re-embed.
- 0.6 → 0.7 minor bump 트리거 (design §9 cascade rule).

• Step 4: Append fb-39b note to fb-39 task spec banner

In tasks/p9/p9-fb-39-retrieval-precision-tuning.md, find the existing > ✅ **Eval foundation 부분 구현 완료.** banner. Append a line:

> - fb-39b (lever 적용 — embedding upgrade): [`tasks/p9/p9-fb-39b-embedding-upgrade.md`](./p9-fb-39b-embedding-upgrade.md) ✅

• Step 5: Add fb-39b row to INDEX

In tasks/INDEX.md, find the fb-39 row. Add a sibling row immediately below:

    - [p9-fb-39b embedding upgrade](p9/p9-fb-39b-embedding-upgrade.md) — ✅ 머지 (2026-05-10) — multilingual-e5-large default

(Adapt format to match neighbor rows.)

• Step 6: Workspace test + clippy gate

cargo test --workspace --no-fail-fast -j 1 2>&1 | tail -10
cargo clippy --workspace --all-targets -- -D warnings 2>&1 | tail -5

-j 1 REQUIRED.

• Step 7: Commit

git add docs/ tasks/
git commit -m "docs(fb-39b): design + HOTFIXES + new task spec + INDEX"

---

Task 5: README + SMOKE walkthrough

Files:

• Modify: README.md

• Modify: docs/SMOKE.md

• Step 1: Update README [models.embedding] section

grep -n "models.embedding\|multilingual-e5-small\|fastembed" README.md | head -5

Locate the [models.embedding] config block in README. Update default values mentioned + add new bullet:

- `model` (default `"multilingual-e5-large"`, fb-39b) — 다국어 sentence embedding 모델. 1024-dim. ONNX (~1.3 GB) 첫 실행 시 fastembed cache (`config.storage.model_dir/fastembed/`) 에 자동 다운로드. `"multilingual-e5-small"` (384 dim) 는 backwards-compat 으로 사용 가능 — TOML 에 명시.
- `dimensions` (default `1024`) — 모델의 embedding 차원. config 와 LanceDB stored dim 불일치 시 검색 결과 0 건 (orphan table). 모델 변경 시 `kebab reset --vector-only && kebab ingest` 로 vector index 재구축 권장.

• Step 2: Append SMOKE walkthrough

Append to docs/SMOKE.md after fb-39 section (or at end if absent):

### Embedding upgrade (fb-39b)

`multilingual-e5-small` 에서 `multilingual-e5-large` 로 업그레이드 시퀀스:

```bash
# 기존 vector index 정리 (orphan table 회피)
kebab --config /tmp/kebab-smoke/config.toml reset --vector-only

# config.toml 의 [models.embedding] 갱신:
#   model = "multilingual-e5-large"
#   dimensions = 1024

# 재-ingest — fastembed 가 첫 실행 시 e5-large ONNX (~1.3 GB) 자동 다운로드.
# 다운로드 시간 + 모든 chunk re-embed 시간 (e5-small 대비 ~3-4×).
kebab --config /tmp/kebab-smoke/config.toml ingest

# fb-39 의 P@k metric 으로 small vs large 비교:
kebab --config /tmp/kebab-smoke/config.toml eval run
```

• Step 3: Workspace test + clippy gate (sanity)

cargo test --workspace --no-fail-fast -j 1 2>&1 | tail -5
cargo clippy --workspace --all-targets -- -D warnings 2>&1 | tail -3

• Step 4: Commit

git add README.md docs/SMOKE.md
git commit -m "docs(fb-39b): README + SMOKE — embedding upgrade walkthrough"

---

Final verification checklist

• cargo test --workspace --no-fail-fast -j 1 green
• cargo clippy --workspace --all-targets -- -D warnings clean
• kebab schema --json | jq .models.embedding_version reflects new model name (after a fresh ingest with new defaults)
• Manual smoke: kebab reset --vector-only && kebab ingest against /tmp/kebab-smoke triggers ONNX download (first run) then completes ingest into the new chunk_embeddings_multilingual-e5-large_1024 table
• README + SMOKE + design + HOTFIXES + fb-39b spec + INDEX all updated
• Post-merge: cut version bump 0.6 → 0.7 + tag (CLAUDE.md Versioning cascade release rule — embedding_version cascade triggers minor bump)