altair823-org/kebab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

feat(fb-39b): embedding upgrade — multilingual-e5-large default #137

Merged
altair823 merged 6 commits from feat/fb-39b-embedding-upgrade into main 2026-05-10 14:53:24 +00:00
Conversation 0 Commits 6 Files Changed 13 +794 -46
altair823 commented 2026-05-10 14:30:04 +00:00
Owner
Copy Link

Summary

  • Default embedding model flipped: multilingual-e5-small (384 dim) → multilingual-e5-large (1024 dim).
  • kebab-embed-local resolve_model gains e5-large arm. e5-small kept for backwards-compat.
  • Config defaults flip + TOML template updated. User TOML pinning small still honored.
  • Cascade: embedding_version mismatch → fb-23 incremental ingest auto-re-embeds all chunks into new (model, dim) LanceDB table on next kebab ingest.
  • Old chunk_embeddings_multilingual-e5-small_384 table becomes orphan — kebab reset --vector-only cleans before re-ingest.
  • bge-m3 deferred (fastembed enum 미포함, UserDefinedEmbeddingModel ONNX path 별도 — fb-39c 후보).

Cascade trigger

Design §9 cascade rule: embedding_version 변경 → 0.6 → 0.7 minor bump. Post-merge: chore: bump version 0.6 → 0.7 + tag.

Design: docs/superpowers/specs/2026-05-10-p9-fb-39b-embedding-upgrade-design.md
Plan: docs/superpowers/plans/2026-05-10-p9-fb-39b-embedding-upgrade.md

Test plan

  • cargo test --workspace --no-fail-fast -j 1 green
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • kebab-embed-local: 3 unit tests (resolve_model arm + check_dim 1024 + dim mismatch)
  • kebab-config: default test asserts new model + dimensions
  • Manual smoke: kebab reset --vector-only && kebab ingest against /tmp/kebab-smoke triggers e5-large ONNX download (first run) + re-embed all chunks

🤖 Generated with Claude Code

## Summary - Default embedding model flipped: multilingual-e5-small (384 dim) → multilingual-e5-large (1024 dim). - kebab-embed-local resolve_model gains e5-large arm. e5-small kept for backwards-compat. - Config defaults flip + TOML template updated. User TOML pinning small still honored. - Cascade: embedding_version mismatch → fb-23 incremental ingest auto-re-embeds all chunks into new (model, dim) LanceDB table on next kebab ingest. - Old chunk_embeddings_multilingual-e5-small_384 table becomes orphan — kebab reset --vector-only cleans before re-ingest. - bge-m3 deferred (fastembed enum 미포함, UserDefinedEmbeddingModel ONNX path 별도 — fb-39c 후보). ## Cascade trigger Design §9 cascade rule: embedding_version 변경 → 0.6 → 0.7 minor bump. Post-merge: chore: bump version 0.6 → 0.7 + tag. Design: docs/superpowers/specs/2026-05-10-p9-fb-39b-embedding-upgrade-design.md Plan: docs/superpowers/plans/2026-05-10-p9-fb-39b-embedding-upgrade.md ## Test plan - [x] cargo test --workspace --no-fail-fast -j 1 green - [x] cargo clippy --workspace --all-targets -- -D warnings clean - [x] kebab-embed-local: 3 unit tests (resolve_model arm + check_dim 1024 + dim mismatch) - [x] kebab-config: default test asserts new model + dimensions - [ ] Manual smoke: `kebab reset --vector-only && kebab ingest` against /tmp/kebab-smoke triggers e5-large ONNX download (first run) + re-embed all chunks 🤖 Generated with Claude Code
altair823 added 4 commits 2026-05-10 14:30:05 +00:00
spec(fb-39b): embedding model upgrade design 2c3461c465 
- multilingual-e5-small (384 dim) → multilingual-e5-large (1024 dim)
- Cascade: embedding_version bump → fb-23 incremental ingest
  re-embeds all chunks
- Migration policy: dim mismatch detection at LanceVectorStore::open
  → error.v1 (code = embedding_dim_mismatch) + hint
  "kebab reset --vector-only && kebab ingest"
- Config defaults flip (model + dimensions). User TOML pinning small
  preserves backwards-compat
- bge-m3 deferred (fastembed enum 미포함, UserDefinedEmbeddingModel
  ONNX path 별도)
- Release trigger: 0.6 → 0.7 minor bump per CLAUDE.md cascade rule

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
plan(fb-39b): embedding upgrade implementation plan d5321701ea 
5 tasks: kebab-embed-local resolve_model arm + check_dim test,
kebab-config defaults + TOML template flip, cross-crate fixture
sweep (likely no-op since most tests use provider=none), docs
(design + HOTFIXES + new task spec + INDEX), README + SMOKE
walkthrough.

Post-merge: 0.6 → 0.7 binary bump per CLAUDE.md cascade rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(embed,config): add multilingual-e5-large + flip default config (fb-39b) 69c94b6692 
Task 1: Add multilingual-e5-large arm to kebab-embed-local::resolve_model with tests for 1024-dim variants and error cases.

Task 2: Flip kebab-config defaults from e5-small (384-dim) to e5-large (1024-dim) across defaults(), test assertions, and TOML template.

All tests pass; clippy clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs(fb-39b): design + HOTFIXES + new task spec + INDEX + README + SMOKE c62a8ff503 
Tasks 4 + 5: comprehensive doc update for embedding upgrade (multilingual-e5-large).

- design §5 + §9: update embedding_model / dimensions references (384 -> 1024)
- HOTFIXES: add fb-39b entry with user re-ingest procedure + backwards-compat notes
- tasks/p9-fb-39b-embedding-upgrade.md: new task spec (completed status)
- INDEX.md: add fb-39b row under RAG quality phase
- fb-39 task banner: append fb-39b link as lever implementation
- README: update config defaults + fastembed model size + embedding field docs
- SMOKE.md: append embedding upgrade verification section with e5-small -> e5-large sequence

Wire schema: no change (additive at config level, new table created by existing code).
Binary version: 0.6.0 -> 0.7.0 (cascade rule: embedding_model change = minor bump).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
altair823 added 1 commit 2026-05-10 14:46:01 +00:00
fix(fb-39b): address PR #137 round 1 review b954e9ce66 
- CI-only embed_model.rs tests updated 384 → 1024 + e5-small → e5-large
  references (incl. file header download size, snapshot dim assert,
  L2 norm comment)
- kebab-embed-local module docs + Cargo.toml description list both
  models (small + large)
- Stale tracing message expanded with both model sizes
- Task spec Post-merge deviation section: record dropped
  embedding_dim_mismatch ErrorV1 + reason (LanceDB (model, dim)
  namespacing makes hard-error redundant)
- Task spec + HOTFIXES version bump 0.6→0.7 corrected to 0.5→0.6
  (current Cargo.toml = 0.5.0; fb-42 0.6 cut deferred per user
  direction)
- HOTFIXES "embedding_version bump 아님" line corrected — cascade rule
  DOES trigger release bump, plus deviation note for the dropped error

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
altair823 added 1 commit 2026-05-10 14:47:52 +00:00
fix(fb-39b): address PR #137 round 2 review 787007172a 
- target_version 0.7.0 → 0.6.0 (current Cargo.toml = 0.5.0;
  embedding_version cascade bumps to 0.6, not 0.7)
- 요약 bullet "0.6 → 0.7" → "0.5 → 0.6" 정정

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
altair823 merged commit a465b71f99 into main 2026-05-10 14:53:24 +00:00
altair823 deleted branch feat/fb-39b-embedding-upgrade 2026-05-10 14:53:25 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
altair823 claude-reviewer-01
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: altair823-org/kebab#137
Delete Branch "feat/fb-39b-embedding-upgrade"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?