refactor(embed): candle provider/crate 제거 — fastembed+ollama로 충분

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Mc6W1fgsrbFKTsqA6P8La
This commit is contained in:
2026-06-24 09:37:28 +00:00
parent 1d32aa645a
commit 040d17f122
17 changed files with 47 additions and 1791 deletions

777
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -11,7 +11,6 @@ members = [
"crates/kebab-search",
"crates/kebab-embed",
"crates/kebab-embed-local",
"crates/kebab-embed-candle",
"crates/kebab-embed-ollama",
"crates/kebab-llm",
"crates/kebab-llm-local",

View File

@@ -30,7 +30,7 @@ P0~P5 직렬. P6~P9 P5 이후 병렬 가능.
## 머지 후 발견된 버그 / 결정 (요약)
- **candle 임베딩 백엔드 다변화** (2026-06-01, Track 1, v0.22.0): `provider = "candle"` opt-in 추가 — 같은 `multilingual-e5-large` 모델을 순수 Rust(candle)로 돌려 듀얼소켓 NUMA 서버의 onnxruntime 48-스레드 double-free 를 회피. `[models.embedding].num_threads`(+env `KEBAB_EMBED_THREADS`)로 CPU 스레드 캡. fastembed default 동작·벡터 불변, `embedding_version` 유지(재색인 0). Phase 0 스파이크 패리티 cosine 1.000000. 상세 HOTFIXES 동일 일자.
- **candle 임베딩 백엔드 제거** (2026-06-24, spine-phase0): `provider = "candle"` 제거 — fastembed(기본) + ollama 두 경로로 충분, candle 은 미사용. `num_threads` 필드는 TOML 하위호환 위해 유지(레거시, 무시). 상세: spine-cuts 계획.
- **config 마이그레이션** (2026-05-31, PR #198): `kebab config migrate` 추가 — 기존 config.toml 에 빠진 섹션을 주석과 함께 채우고 deprecated 정리(멱등·`.bak`·dry-run, 값/주석 보존). `schema_version` 1→2, `init` 도 섹션 주석 포함, doctor 에 `config_migration` 체크. 상세 HOTFIXES 동일 일자.
머지 후 발견된 모든 deviation / hotfix 의 dated 로그는 [tasks/HOTFIXES.md](tasks/HOTFIXES.md). 본 요약은 \"누군가가 인수받을 때 알아두면 시간을 많이 절약하는\" 항목만:
@@ -40,7 +40,7 @@ P0~P5 직렬. P6~P9 P5 이후 병렬 가능.
- **2026-06-04 PP-OCRv5 ONNX Rust 네이티브 OCR** — v0.27.0. `[image.ocr] engine = "paddle-onnx"` 로 PP-OCRv5(검출+인식) ONNX 를 in-process(`ort` =2.0.0-rc.9) 실행 — Python 런타임/원격 호출 없이 큰 페이지 CPU <4초(Ollama vision ~50초 대비). default 는 여전히 `"ollama-vision"`. 후처리(min-area rect/unclip)는 pure-Rust. **함정**: unclip 은 corner 를 centroid 에서 방사 확장하면 안 되고 edge 별 polygon offset 이어야 함(방사 확장 시 wide/short 텍스트 박스 높이가 안 커져 글자 윗부분 잘림 → ㄷ→ㄴ, e2e CER 0.26). 수정 후 CER 0.005. 모델 ONNX 는 `crates/kebab-parse-image/assets/paddleocr-onnx/`(LFS). 자세한 내용: `tasks/HOTFIXES.md` (2026-06-04 PP-OCRv5 ONNX), spec/plan `docs/superpowers/{specs,plans}/2026-06-04-rust-native-ocr-*.md`.
- **2026-06-03 ingest 설정 변경 자동 재색인** — v0.26.2. ingest 산출에 영향 주는 설정(청킹/이미지 OCR·caption/pdf.ocr/`[ingest.code]`)을 변경하면 `--force-reingest` 없이 영향 자산만 자동 재색인. 그 설정들의 결정적 서명(`ingest_config_signature`)을 effective parser_version(skip 비교 + 저장 doc 필드 양쪽)에 폴딩 → 다음 ingest 비교가 mismatch. 비산출 설정(search/rag/ui/log + max_pixels/languages/timeout)은 제외(과도 무효화 회피), doc_id 는 base 로 안정 유지. **업그레이드 후 첫 ingest 는 전 자산 1회 재색인**(저장된 상수 parser_version ≠ 새 composite; embedding 은 V012 캐시 히트). 결과 포맷·CLI·wire 불변(내부 skip 판정 정정). 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 ingest 설정 변경 자동 재색인), spec/plan `docs/superpowers/{specs,plans}/2026-06-03-*invalidation*.md`.
- **2026-06-03 ingest 진행 로그 개선** — v0.26.1. 이미지/PDF + OCR/caption on 볼트 ingest 가 "멈춘 듯" 보이던 문제 해소: TTY 진행바에 현재 파일명 + 느린 phase(ocr/caption/embed)+모델명 + 경과초 `(Ns)` heartbeat, 종료 시 최장 소요 파일 top-5 요약. 신규 wire `asset_phase{idx,total,phase,model}` + `asset_timings.ocr_ms`/`caption_ms`(additive, `ingest_progress.v1` 유지, serde default 0). 이미지·PDF 경로도 `asset_timings` emit(이전 markdown 만). 기본 동작 불변. 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 ingest 진행 로그), spec/plan `docs/superpowers/{specs,plans}/2026-06-03-ingest-log-improve-*.md`.
- **2026-06-03 arctic-embed-l-v2.0 임베더 통합** — v0.26.0. 별칭 제거 후 설명형 query recall 보강(측정 recall@10 130/132, e5 +7). `kebab-embed-candle` 모델 레지스트리화(e5 mean + `snowflake-arctic-embed-l-v2.0` CLS, 모델별 pooling/prefix) + 신규 `kebab-embed-ollama`(`provider="ollama"`, `/api/embed`). config `endpoint: Option<String>` 추가. 기본 e5 유지(opt-in), arctic 전환은 embedding_version cascade → 재색인. candle↔Ollama cosine>0.99 게이트로 pooling/prefix 정확성 고정(`#[ignore]`). 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 arctic), spec `docs/superpowers/specs/2026-06-03-arctic-embedder-spec.md`.
- **2026-06-03 arctic-embed-l-v2.0 임베더 통합** — v0.26.0. 별칭 제거 후 설명형 query recall 보강(측정 recall@10 130/132, e5 +7). 신규 `kebab-embed-ollama`(`provider="ollama"`, `/api/embed`). config `endpoint: Option<String>` 추가. 기본 e5 유지(opt-in), arctic 전환은 embedding_version cascade → 재색인. 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 arctic), spec `docs/superpowers/specs/2026-06-03-arctic-embedder-spec.md`.
- **2026-06-03 doc-side expansion(별칭) 기능 완전 제거** — v0.25.0. 아래 2026-05-31 항목의 색인-시 청크당 LLM 별칭 생성 + 별칭 검색 채널을 **전부 제거**(ROI 음수: cross-lingual 은 e5-large 단독으로 충분, 기여는 설명형 +2 그룹뿐인데 대가가 청크당 색인-시 LLM). `Chunk.aliases`/`expansion.rs`/`IngestExpansionCfg`/alias lexical arm/`expansion_progress` wire kind 제거, 신규 마이그레이션 **V013**`chunk_aliases_fts`+`chunks.aliases` DROP. 별칭 default-off 였어 사용자 체감 0, 기존 KB 도 재색인 불요(잔존 별칭 벡터는 `strip_alias_suffix` graceful 매핑/`reset` 정리). `AssetTimings.expansion_ms` 는 wire 호환 위해 값 0 으로 유지. 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03), spec `docs/superpowers/specs/2026-06-03-remove-doc-expansion-spec.md`.
- **2026-05-31 Phase 2 doc-side expansion 별칭(개별 dense 벡터) + 파생물 캐시(V012)** — v0.21.0 cut. 색인 시 LLM 이 청크별 별칭("같은 의미 다른 표현")을 생성, 줄별 **개별 dense 벡터**(sentinel `{chunk}#alias#N`)로 색인 (묶음 1벡터는 평균화 희석으로 회귀 → 폐기) + boilerplate 청크 skip. `[ingest.expansion]` default off. 측정(나무위키 ~1000 문서 CS corpus): 변형 일관성 14/18 → **16/18**, spread 0.222→0.111, 대조군 false-positive 별칭 무죄. 비용 병목(별칭 18문서 2.5h)은 **파생물 캐시(V012, 청크 내용 해시 키)**로 해소 — 정답 3개 cold 1879s → warm 13s **≈ 145배**, embedding+별칭 LLM 캐싱, version_key cascade 정합. search/ask 가 `kebab.sqlite`+`lancedb` 만으로 동작 → 외부 서버 색인 후 DB 만 복사하는 이식 워크플로 가능. **결정/known limitation**: grounded/refusal 판정이 부분 인용을 grounded 로 오분류(정직한 거부가 false-positive 로 집계) — 별도 개선 후보. stack·svm 설명형 2개 잔존. 자세한 내용: `tasks/HOTFIXES.md` (2026-05-31), 측정: `docs/superpowers/handoffs/2026-05-31-namu-wiki-alias-cache-study.md`.
- **2026-05-29 v0.20.2 dogfood findings + 검색 품질 baseline** — 8-finding 라운드 완료. (1) Ask 응답언어: rag-v3 default (질문 언어 = 답변 언어). (2) eval `--config` facade 패치 로 dogfood KB 직접 eval 가능. (3) 검색 품질 baseline — hybrid hit@3=1.0 / MRR=0.833, lexical hit@3=1.0 / MRR=0.7 (golden 10 query). **O-2 known limitation**: 소형 모델(gemma4:e4b) refusal 메시지의 query 언어 불일치 가능 — 판정은 정상, 표시 문구만 해당. 자세한 내용: `tasks/HOTFIXES.md` (2026-05-29).

View File

@@ -47,7 +47,7 @@ embedding 벡터를 청크 **내용 해시** 로 캐싱한다 (`derivation_cache
### 외부 계산 + 로컬 검색 워크플로
search/ask 는 원본 파일 없이 KB 산출물만으로 동작한다 (청크 본문이 SQLite 에 저장되고 문서 경로는 상대경로로 기록됨). 비싼 색인(임베딩·OCR)을 성능 좋은 머신에서 수행한 뒤(예: Apple Silicon 맥에서 candle Metal GPU), **두 산출물만** 다른 머신(예: NUMA 서버)으로 복사하면 그대로 검색·질문할 수 있다.
search/ask 는 원본 파일 없이 KB 산출물만으로 동작한다 (청크 본문이 SQLite 에 저장되고 문서 경로는 상대경로로 기록됨). 비싼 색인(임베딩·OCR)을 성능 좋은 머신에서 수행한 뒤, **두 산출물만** 다른 머신으로 복사하면 그대로 검색·질문할 수 있다.
**무엇을 복사하나 — `[storage]` 에서 정의된 두 경로:**
@@ -64,7 +64,7 @@ rsync -a <src-data_dir>/kebab.sqlite user@server:<dst-data_dir>/
rsync -a <src-data_dir>/lancedb/ user@server:<dst-data_dir>/lancedb/
```
조건: **양쪽 동일 `kebab` 버전 + 동일 임베딩 모델/차원** (`[models.embedding].model`·`dimensions`). provider 는 달라도 됨 (예: `candle`/Metal ↔ 서버 `candle`/CPU 또는 `fastembed` — 같은 모델이면 벡터 호환). 복사는 반드시 ingest 가 돌지 않을 때.
조건: **양쪽 동일 `kebab` 버전 + 동일 임베딩 모델/차원** (`[models.embedding].model`·`dimensions`). provider 는 달라도 됨 (예: `fastembed``ollama` — 같은 모델이면 벡터 호환). 복사는 반드시 ingest 가 돌지 않을 때.
### 멀티미디어 색인
@@ -123,23 +123,16 @@ root = "~/KnowledgeBase" # 색인할 폴더. 절대 / tilde / env / 상대 경
# trust_level = "secondary" # 낮은 신뢰 출처 — `--trust-min primary` 로 배제 가능.
[models.embedding]
provider = "fastembed" # "fastembed"(기본, onnxruntime) / "candle"(순수 Rust)
# / "ollama"(원격 HTTP) / "none"(lexical-only).
# candle 는 같은 모델·같은 벡터를 순수 Rust 로 돌려
# NUMA 서버의 onnxruntime 48-스레드 double-free 를 피하는
# opt-in 백엔드 (e5 는 재색인 불필요).
provider = "fastembed" # "fastembed"(기본, onnxruntime) / "ollama"(원격 HTTP)
# / "none"(lexical-only).
model = "multilingual-e5-large" # 다국어 sentence embedding (1024-dim).
# 첫 ingest 시 ONNX (~1.3GB) 자동 다운로드.
# candle provider 는 safetensors (~2GB) 다운로드.
# candle/ollama 는 "snowflake-arctic-embed-l-v2.0"
# ollama 는 "snowflake-arctic-embed-l-v2.0"
# (설명형 query 의 recall 보강) 도 지원 — 아래 참고.
dimensions = 1024 # config 와 LanceDB stored dim 불일치 시 검색 0건.
num_threads = 0 # candle 전용 CPU 스레드 캡 (0=auto=#cores).
# env KEBAB_EMBED_THREADS 가 우선. NUMA 노드 바인딩은
# numactl 과 조합. fastembed provider 는 무시.
# endpoint = "http://127.0.0.1:11434" # provider="ollama" 전용 HTTP endpoint.
# 생략 시 [models.llm].endpoint 로 폴백.
# fastembed/candle provider 는 무시.
# fastembed provider 는 무시.
```
**arctic-embed-l-v2.0 (설명형 query recall 보강)**: 기본 e5-large 대신
@@ -147,13 +140,7 @@ Snowflake `arctic-embed-l-v2.0` 임베더를 쓸 수 있다 (1024-dim, opt-in).
설명형/약어/영문 용어 query 의 recall@10 이 e5 대비 향상됐다. 두 경로:
```toml
# (A) candle 백엔드 — 순수 Rust, in-process (NUMA 안전, Metal GPU 가능):
[models.embedding]
provider = "candle"
model = "snowflake-arctic-embed-l-v2.0" # CLS pooling, query 에 "query: " 접두어
# (문서는 무접두어). safetensors ~2GB 다운로드.
# (B) ollama 백엔드 — 원격/로컬 Ollama 데몬에 위임 (POST /api/embed):
# ollama 백엔드 — 원격/로컬 Ollama 데몬에 위임 (POST /api/embed):
[models.embedding]
provider = "ollama"
model = "snowflake-arctic-embed2" # Ollama 모델 태그 (ollama pull 필요)
@@ -164,21 +151,6 @@ endpoint = "http://127.0.0.1:11434" # 생략 시 [models.llm].endpoint
> 벡터도 다름). 기존 e5 KB 와 혼용 불가 — 전환 시 **재색인** 필요 (`kebab reset`
> 후 재 ingest). 기본값은 e5 라 기존 사용자는 영향 없음.
**Apple Silicon GPU 가속 (candle / macOS)**: M-시리즈 맥에서 candle 임베딩을
GPU(Metal)로 돌리면 CPU 대비 대용량 ingest 가 크게 빨라진다. 빌드 또는 설치 시
`embed_metal` feature 를 켠다:
```bash
# 빌드만:
cargo build --release --features embed_metal
# 전역 설치 (~/.cargo/bin/kebab):
cargo install --path crates/kebab-cli --features embed_metal --locked
```
벡터는 CPU candle 과 동일 모델이라 호환되므로, 맥에서 GPU 로 색인한
`kebab.sqlite` + `lancedb/` 를 그대로 Linux 서버(CPU candle)로 복사해 질의할 수
있다. 색인 로그에 `candle device = Metal (GPU)` 가 보이면 GPU 사용 중. metal
feature 는 macOS 전용 (Linux/서버는 기본 CPU 빌드).
```toml

View File

@@ -18,7 +18,6 @@ kebab-store-vector = { path = "../kebab-store-vector" }
kebab-search = { path = "../kebab-search" }
kebab-embed = { path = "../kebab-embed" }
kebab-embed-local = { path = "../kebab-embed-local" }
kebab-embed-candle = { path = "../kebab-embed-candle" }
kebab-embed-ollama = { path = "../kebab-embed-ollama" }
kebab-llm = { path = "../kebab-llm" }
kebab-llm-local = { path = "../kebab-llm-local" }
@@ -95,8 +94,6 @@ reqwest = { version = "0.12", default-features = false, features = ["blocki
# disable path 없음; 이 feature 는 spec §6.3 명시를 honor 하는 role 만.
default = ["fts_korean_morphological"]
fts_korean_morphological = []
# opt-in (macOS): candle embedder runs on the Apple Silicon GPU. See kebab-embed-candle.
embed_metal = ["kebab-embed-candle/metal"]
[lints]
workspace = true

View File

@@ -41,7 +41,6 @@ use kebab_core::{
Answer, DocumentStore, Embedder, ExtractContext, Extractor, IndexVersion, LanguageModel,
MediaType, Retriever, SearchHit, SearchMode, SearchOpts, SearchQuery, VectorStore,
};
use kebab_embed_candle::CandleEmbedder;
use kebab_embed_local::FastembedEmbedder;
use kebab_embed_ollama::OllamaEmbedder;
use kebab_llm_local::OllamaLanguageModel;
@@ -607,28 +606,23 @@ impl App {
if let Some(e) = self.embedder.get() {
return Ok(Some(e.clone()));
}
// Provider branch (Track 1 spec §3 + arctic-embedder spec). The
// `embeddings_disabled()` check above already handled `"none"`; here we
// route the live providers. `fastembed`/`onnx`/(empty) keep the default
// onnxruntime path (vectors unchanged — `embedding_version` is
// preserved); `candle` selects the pure-Rust NUMA-safe backend (e5 or
// arctic via its model registry); `ollama` offloads to a remote
// `/api/embed` daemon.
// Provider branch (arctic-embedder spec). The `embeddings_disabled()`
// check above already handled `"none"`; here we route the live
// providers. `fastembed`/`onnx`/(empty) keep the default onnxruntime
// path (vectors unchanged — `embedding_version` is preserved); `ollama`
// offloads to a remote `/api/embed` daemon.
let provider = self.config.models.embedding.provider.as_str();
let emb: Arc<dyn Embedder + Send + Sync> = match provider {
"fastembed" | "onnx" | "" => Arc::new(
FastembedEmbedder::new(&self.config).context("kb-app: load FastembedEmbedder")?,
),
"candle" => Arc::new(
CandleEmbedder::new(&self.config).context("kb-app: load CandleEmbedder")?,
),
"ollama" => Arc::new(
OllamaEmbedder::new(&self.config).context("kb-app: load OllamaEmbedder")?,
),
other => {
return Err(anyhow!(
"kb-app: unknown embedding provider {other:?}; expected one of \
`fastembed` (default), `candle`, `ollama`, or `none` (lexical-only)"
`fastembed` (default), `ollama`, or `none` (lexical-only)"
));
}
};

View File

@@ -51,10 +51,5 @@ tempfile = { workspace = true }
rusqlite = { workspace = true }
time = { workspace = true }
[features]
# opt-in (macOS): build the `kebab` binary with candle on the Apple Silicon GPU.
# cargo build --release --features embed_metal
embed_metal = ["kebab-app/embed_metal"]
[lints]
workspace = true

View File

@@ -654,14 +654,9 @@ fn run(cli: &Cli) -> anyhow::Result<()> {
let mode = progress::ProgressMode::from_flags(cli.json, cli.quiet, plain_env);
// Surface the active embedding backend/device on the terminal so the
// user sees it without grepping kb.log (the per-device tracing line
// only lands in the log file at --verbose). Suppressed under
// --json/--quiet. The Metal note reflects the build (`embed_metal`);
// the confirmed runtime device is in kb.log (`candle device = ...`).
// user sees it without grepping kb.log. Suppressed under --json/--quiet.
if !cli.json && !cli.quiet {
let backend = match cfg.models.embedding.provider.as_str() {
"candle" if cfg!(feature = "embed_metal") => "candle (Metal/GPU 빌드)",
"candle" => "candle (CPU, 순수 Rust)",
"fastembed" | "onnx" | "" => "fastembed (onnxruntime)",
"none" => "비활성 (lexical-only)",
other => other,

View File

@@ -255,26 +255,24 @@ impl NliCfg {
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct EmbeddingModelCfg {
/// `fastembed` (default, onnxruntime), `candle` (pure-Rust, NUMA-safe),
/// or `ollama` (remote HTTP embedding endpoint). `none` disables
/// embeddings (lexical-only). Unknown values error at embedder
/// construction.
/// `fastembed` (default, onnxruntime) or `ollama` (remote HTTP embedding
/// endpoint). `none` disables embeddings (lexical-only). Unknown values
/// error at embedder construction.
pub provider: String,
pub model: String,
pub version: String,
pub dimensions: usize,
pub batch_size: usize,
/// Cap on the CPU worker threads the `candle` provider spins up
/// (sizes the global rayon pool; env `KEBAB_EMBED_THREADS` overrides).
/// `0` = auto (rayon default = #cores). Lever to sidestep the
/// onnxruntime 48-thread NUMA double-free; ignored by the `fastembed`
/// provider. Defaulted on load so pre-0.22 config files still parse.
/// Legacy field — previously used by the removed `candle` provider to cap
/// CPU worker threads. Retained for backward-compatible TOML parsing
/// (`num_threads = 0` in old config files must not error). Ignored by all
/// current providers.
#[serde(default)]
pub num_threads: u32,
/// HTTP endpoint for the `ollama` embedding provider (e.g.
/// `"http://127.0.0.1:11434"`). `None` (or a missing key in TOML) means
/// "fall back to `models.llm.endpoint`" — same convention as the OCR /
/// vision endpoints. Ignored by the `fastembed` / `candle` providers.
/// vision endpoints. Ignored by the `fastembed` provider.
/// Defaulted on load so pre-0.26 config files still parse.
#[serde(default)]
pub endpoint: Option<String>,

View File

@@ -99,9 +99,9 @@ fn key_comment(path: &str) -> Option<&'static str> {
"workspace.root" => "색인 루트. 절대/~/${VAR}/상대(=이 파일 기준).",
"workspace.exclude" => "denylist glob.",
"storage.copy_threshold_mb" => "이 크기(MB) 초과 파일은 사본 대신 참조.",
"models.embedding.provider" => "fastembed | candle | ollama | none.",
"models.embedding.provider" => "fastembed | ollama | none.",
"models.embedding.dimensions" => "모델 출력 차원. 틀리면 검색 0건.",
"models.embedding.num_threads" => "candle 전용 CPU 스레드 cap(0=auto).",
"models.embedding.num_threads" => "레거시 필드 (deprecated, 무시됨).",
"models.embedding.endpoint" => "ollama provider 시 HTTP. 비우면 llm.endpoint fallback.",
"models.llm.request_timeout_secs" => "단일 HTTP 상한. 0=즉시실패(비활성화 아님).",
"ingest.max_parallel_extractors" => "동시 extractor 수.",

View File

@@ -1,50 +0,0 @@
[package]
name = "kebab-embed-candle"
version = { workspace = true }
edition = { workspace = true }
rust-version = { workspace = true }
license = { workspace = true }
repository = { workspace = true }
description = "Pure-Rust candle adapter implementing kb_core::Embedder (multilingual-e5-large, NUMA-safe thread cap)"
[dependencies]
kebab-core = { path = "../kebab-core" }
kebab-config = { path = "../kebab-config" }
# candle stack — pinned to the workspace-locked crates.io release (0.10.x),
# same versions the Phase 0 spike compiled so build artifacts are reused.
candle-core = "0.10.2"
candle-nn = "0.10.2"
candle-transformers = "0.10.2"
tokenizers = "0.21"
hf-hub = { version = "0.4", features = ["ureq"] }
serde_json = { workspace = true }
# Thread cap: a one-shot global rayon pool sizes candle's CPU threads
# (the Phase 0 spike proved RAYON_NUM_THREADS caps candle), so a NUMA host
# can keep onnxruntime's hard-coded 48-intra-op heap corruption at bay.
rayon = "1"
anyhow = { workspace = true }
tracing = { workspace = true }
[features]
# opt-in: run candle on the Apple Silicon GPU (Metal). macOS-only — the build
# enables candle's metal backend and `select_device()` picks Metal (CPU fallback
# on failure). Lets an M-series Mac ingest e5-large on GPU (10×+ vs CPU); the
# resulting vectors are cross-compatible with the CPU path (same model), so the
# Linux server can serve queries on CPU candle.
metal = ["candle-core/metal", "candle-nn/metal", "candle-transformers/metal"]
[dev-dependencies]
# Integration-test binaries can only see the library's public API + these,
# not the library's own (non-dev) dependencies — so rayon/kebab-config/kebab-core
# are repeated here for tests/parity.rs and tests/thread_cap.rs.
kebab-embed-local = { path = "../kebab-embed-local" }
# arctic↔Ollama parity test drives the real Ollama adapter for the reference
# vectors (tests/arctic_ollama_parity.rs, `#[ignore]` — live Ollama).
kebab-embed-ollama = { path = "../kebab-embed-ollama" }
kebab-config = { path = "../kebab-config" }
kebab-core = { path = "../kebab-core" }
rayon = "1"
tempfile = { workspace = true }
[lints]
workspace = true

View File

@@ -1,619 +0,0 @@
//! `kebab-embed-candle` — [`CandleEmbedder`], a pure-Rust (candle)
//! implementation of [`Embedder`](kebab_core::Embedder).
//!
//! Runs an XLM-RoBERTa-large embedding model through `candle`
//! (`candle-transformers`' XLM-RoBERTa) instead of onnxruntime. Two models
//! are wired through a small **registry** ([`MODEL_REGISTRY`]):
//!
//! * `multilingual-e5-large` — the same weights the default
//! [`FastembedEmbedder`](kebab_embed_local) uses (mean pooling,
//! `query: `/`passage: ` prefixes). candle is the NUMA-safe drop-in:
//! fastembed 4.9's onnxruntime hard-codes 48 intra-op threads, which
//! corrupts the heap (double-free) on dual-socket NUMA hosts. candle's
//! CPU backend sizes its threads off the global rayon pool, so a one-shot
//! [`rayon::ThreadPoolBuilder`] cap (config `num_threads` / env
//! `KEBAB_EMBED_THREADS`) keeps the worker count NUMA-safe.
//! * `snowflake-arctic-embed-l-v2.0` — Snowflake's arctic-embed v2.0
//! (CLS pooling, `query: ` on queries / no prefix on documents). Same
//! XLM-RoBERTa-large architecture, dim 1024, so it rides the exact same
//! tokenize → forward → L2 pipeline; only the pooling step and prefixes
//! differ (both keyed off the per-model [`EmbedModelSpec`]).
//!
//! Output parity with the onnxruntime path (for e5) was proven by the
//! Phase 0 spike (cosine 1.000000); the arctic path's pooling/prefix
//! correctness is pinned by an `#[ignore]`d cosine>0.99 cross-check against
//! Ollama's `snowflake-arctic-embed2` (see `tests/arctic_ollama_parity.rs`).
//! The shared pipeline:
//!
//! 1. instruction prefix per [`EmbedModelSpec`] (query/doc);
//! 2. tokenize (max_len 512, batch-longest padding, special tokens);
//! 3. XLM-RoBERTa forward on the selected [`Device`];
//! 4. pooling — mean (attention-mask-weighted) or CLS (first token);
//! 5. L2 normalization.
//!
//! Model files (`config.json`, `tokenizer.json`, `model.safetensors`) are
//! fetched via `hf-hub` into `{config.storage.model_dir}/candle/` (hf-hub's
//! cache layout namespaces by repo, so e5 and arctic never collide).
//!
//! This crate is **opt-in** (`config.models.embedding.provider = "candle"`);
//! the default provider stays `fastembed`. See
//! `docs/superpowers/specs/2026-06-01-embed-candle-track-spec.md` and
//! `docs/superpowers/specs/2026-06-03-arctic-embedder-spec.md`.
use std::sync::Mutex;
use anyhow::{Context, Result};
use candle_core::{DType, Device, Tensor};
use candle_nn::VarBuilder;
use candle_transformers::models::xlm_roberta::{Config as XlmConfig, XLMRobertaModel};
use kebab_config::{Config, expand_path};
use kebab_core::{Embedder, EmbeddingInput, EmbeddingKind, EmbeddingModelId, EmbeddingVersion};
use tokenizers::{PaddingParams, PaddingStrategy, Tokenizer, TruncationParams};
/// Subdirectory under `config.storage.model_dir` where the candle adapter
/// caches safetensors + tokenizer. Mirrors `kebab-embed-local`'s
/// `fastembed/` subdir so the two backends never collide.
const CANDLE_CACHE_SUBDIR: &str = "candle";
/// Token truncation length (both e5 and arctic-embed-l-v2.0 train at 512).
const MAX_LEN: usize = 512;
/// Env var that overrides `config.models.embedding.num_threads`. Read once in
/// [`CandleEmbedder::new`]; `0`/unset/unparseable means "leave rayon default".
const ENV_EMBED_THREADS: &str = "KEBAB_EMBED_THREADS";
/// Pooling strategy over the model's last hidden state. Keyed per-model by
/// [`EmbedModelSpec::pooling`] — e5 is mean, arctic is CLS.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Pooling {
/// Attention-mask-weighted mean over all tokens (e5 / sentence-transformers
/// `pooling_mode_mean_tokens`).
Mean,
/// First token (`<s>`/`[CLS]`) hidden state (arctic-embed v2.0 —
/// `1_Pooling/config.json` has `pooling_mode_cls_token: true`).
Cls,
}
/// One supported embedding model: the HF repo candle downloads, the pooling
/// strategy, and the e5-style instruction prefixes. [`MODEL_REGISTRY`] maps a
/// `config.models.embedding.model` value to one of these.
#[derive(Clone, Copy, Debug)]
pub struct EmbedModelSpec {
/// The short `config.models.embedding.model` value that selects this spec.
pub name: &'static str,
/// HuggingFace repo id candle fetches `config.json` / `tokenizer.json` /
/// `model.safetensors` from.
pub hf_repo: &'static str,
/// Pooling over the last hidden state.
pub pooling: Pooling,
/// Prefix prepended to **query** inputs before tokenization.
pub query_prefix: &'static str,
/// Prefix prepended to **document** inputs before tokenization (arctic
/// uses `""` — documents are embedded raw).
pub doc_prefix: &'static str,
/// Expected embedding dimension (model hidden size).
pub dim: usize,
/// Suffix folded into `model_version` so switching **to** this model
/// triggers the `embedding_version` cascade even if the operator forgets
/// to bump `config.version`. `None` keeps the bare `config.version` — used
/// by e5 so candle-e5 and fastembed-e5 report the *same* version and stay
/// interchangeable (the NUMA drop-in invariant — Phase 0 cosine 1.0).
pub version_tag: Option<&'static str>,
}
/// The models the candle adapter can load. Adding a model = one entry here
/// (plus, for a non-XLM-R architecture, a new forward path — both current
/// entries are XLM-RoBERTa-large so they share everything but pooling/prefix).
static MODEL_REGISTRY: &[EmbedModelSpec] = &[
EmbedModelSpec {
name: "multilingual-e5-large",
hf_repo: "intfloat/multilingual-e5-large",
pooling: Pooling::Mean,
query_prefix: "query: ",
doc_prefix: "passage: ",
dim: 1024,
version_tag: None,
},
EmbedModelSpec {
name: "snowflake-arctic-embed-l-v2.0",
hf_repo: "Snowflake/snowflake-arctic-embed-l-v2.0",
pooling: Pooling::Cls,
query_prefix: "query: ",
doc_prefix: "",
dim: 1024,
version_tag: Some("arctic-cls"),
},
];
/// Look up a model spec by `config.models.embedding.model`. Accepts either the
/// short `name` or the full `hf_repo` id (mirrors the old e5 guard, which
/// accepted both `multilingual-e5-large` and `intfloat/multilingual-e5-large`).
pub(crate) fn lookup_spec(model: &str) -> Option<&'static EmbedModelSpec> {
MODEL_REGISTRY
.iter()
.find(|s| s.name == model || s.hf_repo == model)
}
/// Comma-separated list of supported model names, for the
/// unsupported-model error message.
fn supported_models() -> String {
MODEL_REGISTRY
.iter()
.map(|s| s.name)
.collect::<Vec<_>>()
.join("`, `")
}
/// Pure-Rust candle adapter. Construct via [`CandleEmbedder::new`]; the
/// constructor downloads the model on first use, so share one instance.
pub struct CandleEmbedder {
// candle's `forward` is `&self`, but `XLMRobertaModel` is not guaranteed
// `Sync`; the `Mutex` both supplies that bound and serializes inference
// (callers batch sequentially anyway — same rationale as
// `FastembedEmbedder`).
model: Mutex<XLMRobertaModel>,
tokenizer: Tokenizer,
device: Device,
/// The resolved model spec (pooling + prefixes) — drives `embed` and
/// `embed_batch`.
spec: &'static EmbedModelSpec,
model_id: EmbeddingModelId,
version: EmbeddingVersion,
dimensions: usize,
batch_size: usize,
}
impl CandleEmbedder {
/// Build an embedder from `Config`. Resolves the model spec from
/// `config.models.embedding.model`, applies the NUMA thread cap, fetches
/// the model into `{model_dir}/candle/`, and validates that the model's
/// hidden size matches `config.models.embedding.dimensions` before
/// returning.
pub fn new(config: &Config) -> Result<Self> {
// 1. NUMA thread cap. env `KEBAB_EMBED_THREADS` wins over the config
// field; `0`/unset leaves rayon's default. `build_global` errors if
// the pool was already initialized — intentionally ignored so a
// second embedder (or a prior rayon user) is a no-op, not a failure.
let n_threads = std::env::var(ENV_EMBED_THREADS)
.ok()
.and_then(|v| v.parse::<usize>().ok())
.unwrap_or(config.models.embedding.num_threads as usize);
if n_threads > 0 {
if apply_thread_cap(n_threads) {
tracing::info!(
target: "kebab-embed-candle",
num_threads = n_threads,
"capped global rayon pool for candle CPU backend"
);
} else {
tracing::debug!(
target: "kebab-embed-candle",
requested = n_threads,
"global rayon pool already initialized; thread cap not applied"
);
}
}
// 1b. Model registry lookup. If the operator configured a model the
// candle adapter doesn't know, fail fast (BEFORE the ~2GB
// download) — never silently download one model and then label its
// vectors with another name via `model_id()`, which would mislabel
// `embedding_version` and corrupt a mixed index.
let want = config.models.embedding.model.as_str();
let spec = lookup_spec(want).ok_or_else(|| {
anyhow::anyhow!(
"candle provider supports the models `{}`, but \
config.models.embedding.model = '{want}'. Use provider=fastembed \
for other models, or pick a supported one.",
supported_models()
)
})?;
// 2. Resolve `{data_dir}/models/candle/` exactly like the fastembed
// adapter resolves its own subdir.
let data_dir = expand_path(&config.storage.data_dir, "");
let model_dir = expand_path(&config.storage.model_dir, &data_dir.to_string_lossy());
let cache_dir = model_dir.join(CANDLE_CACHE_SUBDIR);
std::fs::create_dir_all(&cache_dir)
.with_context(|| format!("create candle cache dir {}", cache_dir.display()))?;
let device = select_device();
// 3. Fetch model files via hf-hub into the candle cache.
tracing::info!(
target: "kebab-embed-candle",
cache_dir = %cache_dir.display(),
model = spec.hf_repo,
pooling = ?spec.pooling,
"loading candle embedding model (first run downloads ~2GB safetensors)"
);
let api = hf_hub::api::sync::ApiBuilder::new()
.with_cache_dir(cache_dir.clone())
.build()
.context("kb-embed-candle: build hf-hub api")?;
let repo = api.model(spec.hf_repo.to_string());
let config_path = repo.get("config.json").context("download config.json")?;
let tokenizer_path = repo
.get("tokenizer.json")
.context("download tokenizer.json")?;
let weights_path = repo
.get("model.safetensors")
.context("download model.safetensors")?;
// 4. Build the candle XLM-RoBERTa model.
let cfg_json = std::fs::read_to_string(&config_path)
.with_context(|| format!("read {}", config_path.display()))?;
let cfg: XlmConfig =
serde_json::from_str(&cfg_json).context("kb-embed-candle: parse XLM-R config")?;
// Validate dim BEFORE building the model so a misconfigured
// `dimensions` fails cheaply (matches FastembedEmbedder's contract).
check_dim(cfg.hidden_size, config.models.embedding.dimensions)?;
let vb = unsafe {
VarBuilder::from_mmaped_safetensors(&[weights_path], DType::F32, &device)
.context("kb-embed-candle: mmap safetensors")?
};
let model =
XLMRobertaModel::new(&cfg, vb).context("kb-embed-candle: build XLMRobertaModel")?;
let mut tokenizer = Tokenizer::from_file(&tokenizer_path)
.map_err(|e| anyhow::anyhow!("kb-embed-candle: load tokenizer: {e}"))?;
tokenizer
.with_padding(Some(PaddingParams {
strategy: PaddingStrategy::BatchLongest,
..Default::default()
}))
.with_truncation(Some(TruncationParams {
max_length: MAX_LEN,
..Default::default()
}))
.map_err(|e| anyhow::anyhow!("kb-embed-candle: set truncation: {e}"))?;
// model_version: fold the model tag in for non-e5 models so a switch
// triggers the embedding_version cascade; e5 keeps the bare
// config.version to stay interchangeable with fastembed-e5.
let version = match spec.version_tag {
Some(tag) => {
EmbeddingVersion(format!("{}+{}", config.models.embedding.version, tag))
}
None => EmbeddingVersion(config.models.embedding.version.clone()),
};
tracing::info!(
target: "kebab-embed-candle",
dimensions = cfg.hidden_size,
layers = cfg.num_hidden_layers,
model = spec.name,
"candle embedding model loaded"
);
Ok(Self {
model: Mutex::new(model),
tokenizer,
device,
spec,
model_id: EmbeddingModelId(config.models.embedding.model.clone()),
version,
dimensions: cfg.hidden_size,
batch_size: config.models.embedding.batch_size.max(1),
})
}
/// Embed one batch of **already-prefixed** strings (the per-model prefix
/// is applied by the caller [`CandleEmbedder::embed`]) through the candle
/// pipeline: tokenize → forward → pool (mean|CLS) → L2.
fn embed_batch(&self, prefixed: &[String]) -> Result<Vec<Vec<f32>>> {
let encodings = self
.tokenizer
.encode_batch(prefixed.to_vec(), true)
.map_err(|e| anyhow::anyhow!("kb-embed-candle: encode_batch: {e}"))?;
let bsz = encodings.len();
// `embed` already returns early on empty input and `.chunks()` never
// yields an empty slice, so this is currently unreachable — but guard
// the index so a future refactor can't turn it into a panic.
let Some(first) = encodings.first() else {
return Ok(Vec::new());
};
let seq = first.get_ids().len();
let mut ids = Vec::with_capacity(bsz * seq);
let mut mask = Vec::with_capacity(bsz * seq);
for enc in &encodings {
ids.extend(enc.get_ids().iter().copied());
mask.extend(enc.get_attention_mask().iter().map(|&m| m as f32));
}
let input_ids = Tensor::from_vec(ids, (bsz, seq), &self.device)?;
let attn_f32 = Tensor::from_vec(mask, (bsz, seq), &self.device)?;
let token_type_ids = input_ids.zeros_like()?;
let hidden = {
let guard = self
.model
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
// forward: (input_ids, attention_mask, token_type_ids, past,
// encoder_hidden, encoder_mask)
guard.forward(&input_ids, &attn_f32, &token_type_ids, None, None, None)?
};
// Pooling — per the model spec.
let pooled = match self.spec.pooling {
Pooling::Mean => {
// attention-mask-weighted mean pooling
let mask3 = attn_f32.unsqueeze(2)?; // (b, seq, 1)
let summed = hidden.broadcast_mul(&mask3)?.sum(1)?; // (b, hidden)
// counts ≥ 1 always: every input is prefixed AND special
// tokens are added (encode_batch(_, true)), so no row has an
// all-zero mask. If that invariant ever breaks, broadcast_div
// would emit NaN vectors.
let counts = mask3.sum(1)?; // (b, 1)
summed.broadcast_div(&counts)?
}
Pooling::Cls => {
// CLS pooling: the first token's hidden state. arctic-embed
// v2.0 prepends `<s>` (the XLM-R BOS/CLS) at index 0, so
// `hidden[:, 0, :]` is the sentence embedding.
hidden.narrow(1, 0, 1)?.squeeze(1)? // (b, hidden)
}
};
// L2 normalize
let norm = pooled.sqr()?.sum_keepdim(1)?.sqrt()?;
let normalized = pooled.broadcast_div(&norm)?;
// `.contiguous()` before host copy: broadcast ops can leave a strided
// view, which `to_vec2` rejects on the Metal backend (CPU tolerates it).
Ok(normalized.contiguous()?.to_vec2::<f32>()?)
}
}
impl Embedder for CandleEmbedder {
fn model_id(&self) -> EmbeddingModelId {
self.model_id.clone()
}
fn model_version(&self) -> EmbeddingVersion {
self.version.clone()
}
fn dimensions(&self) -> usize {
self.dimensions
}
fn embed(&self, inputs: &[EmbeddingInput<'_>]) -> Result<Vec<Vec<f32>>> {
if inputs.is_empty() {
return Ok(Vec::new());
}
// Per-model instruction prefix BEFORE tokenization (same convention as
// FastembedEmbedder for e5; arctic uses `query: `/no-prefix).
let prefixed: Vec<String> = inputs.iter().map(|i| prefix_input(self.spec, i)).collect();
let mut out: Vec<Vec<f32>> = Vec::with_capacity(prefixed.len());
for chunk in prefixed.chunks(self.batch_size) {
let batch = self.embed_batch(chunk)?;
for v in &batch {
if v.len() != self.dimensions {
anyhow::bail!(
"candle returned vector of length {} but adapter expects {}",
v.len(),
self.dimensions
);
}
}
out.extend(batch);
}
debug_assert_eq!(out.len(), inputs.len());
Ok(out)
}
}
/// Build the prefixed string for one [`EmbeddingInput`] using the model spec.
/// Free function so a unit test can pin the format without loading the model.
/// For e5 this is byte-identical to `kebab-embed-local`'s `prefix_input` — the
/// two backends MUST agree there or their vectors diverge.
fn prefix_input(spec: &EmbedModelSpec, input: &EmbeddingInput<'_>) -> String {
match input.kind {
EmbeddingKind::Document => format!("{}{}", spec.doc_prefix, input.text),
EmbeddingKind::Query => format!("{}{}", spec.query_prefix, input.text),
}
}
/// Select the compute device. Built with the `metal` feature (Apple Silicon
/// GPU), try Metal and fall back to CPU on failure; otherwise CPU. Metal only
/// compiles/runs on macOS — the Linux server builds the CPU path. Embedding
/// vectors are model-defined, so Metal-produced and CPU-produced embeddings
/// are cross-compatible (a Mac can ingest on GPU, the server query on CPU).
fn select_device() -> Device {
#[cfg(feature = "metal")]
{
match Device::new_metal(0) {
Ok(d) => {
tracing::info!(target: "kebab-embed-candle", "candle device = Metal (GPU)");
return d;
}
Err(e) => {
tracing::warn!(
target: "kebab-embed-candle",
error = %e,
"Metal device unavailable; falling back to CPU"
);
}
}
}
tracing::info!(target: "kebab-embed-candle", "candle device = CPU");
Device::Cpu
}
/// Apply a one-shot global rayon thread cap (the NUMA-safety lever). Returns
/// `true` if this call set the pool, `false` if it was already initialized
/// (cap not applied) or `n_threads == 0`. `#[doc(hidden)] pub` so the
/// thread-cap test can drive it without loading the 2GB model.
#[doc(hidden)]
pub fn apply_thread_cap(n_threads: usize) -> bool {
if n_threads == 0 {
return false;
}
rayon::ThreadPoolBuilder::new()
.num_threads(n_threads)
.build_global()
.is_ok()
}
/// Compare model hidden size against the configured dim. Extracted so a unit
/// test can exercise the error branch without loading the model.
pub(crate) fn check_dim(model_dim: usize, cfg_dim: usize) -> Result<()> {
if model_dim != cfg_dim {
anyhow::bail!(
"dimension mismatch: model={model_dim}, config={cfg_dim}; \
update `config.models.embedding.dimensions` to match the model \
(or pick a different model)."
);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn e5_spec() -> &'static EmbedModelSpec {
lookup_spec("multilingual-e5-large").expect("e5 in registry")
}
fn arctic_spec() -> &'static EmbedModelSpec {
lookup_spec("snowflake-arctic-embed-l-v2.0").expect("arctic in registry")
}
// ── registry ─────────────────────────────────────────────────────
#[test]
fn registry_resolves_e5_by_name_and_hf_repo() {
assert_eq!(
lookup_spec("multilingual-e5-large").map(|s| s.name),
Some("multilingual-e5-large")
);
assert_eq!(
lookup_spec("intfloat/multilingual-e5-large").map(|s| s.name),
Some("multilingual-e5-large")
);
}
#[test]
fn registry_resolves_arctic_and_its_pooling_is_cls() {
let s = arctic_spec();
assert_eq!(s.name, "snowflake-arctic-embed-l-v2.0");
assert_eq!(s.hf_repo, "Snowflake/snowflake-arctic-embed-l-v2.0");
assert_eq!(s.pooling, Pooling::Cls);
assert_eq!(s.dim, 1024);
assert_eq!(s.version_tag, Some("arctic-cls"));
}
#[test]
fn registry_e5_is_mean_pooling_no_version_tag() {
let s = e5_spec();
assert_eq!(s.pooling, Pooling::Mean);
assert_eq!(s.version_tag, None);
}
#[test]
fn registry_rejects_unknown_model() {
assert!(lookup_spec("multilingual-e5-small").is_none());
}
// ── prefix_input ─────────────────────────────────────────────────
// e5 prefixes MUST match kebab-embed-local::prefix_input or candle vs
// fastembed parity breaks; arctic uses query-only prefixing.
#[test]
fn e5_prefix_document_uses_passage() {
let input = EmbeddingInput {
text: "hello world",
kind: EmbeddingKind::Document,
};
assert_eq!(prefix_input(e5_spec(), &input), "passage: hello world");
}
#[test]
fn e5_prefix_query_uses_query() {
let input = EmbeddingInput {
text: "hello world",
kind: EmbeddingKind::Query,
};
assert_eq!(prefix_input(e5_spec(), &input), "query: hello world");
}
#[test]
fn arctic_prefix_query_uses_query_doc_is_bare() {
let doc = EmbeddingInput {
text: "후입선출 자료구조",
kind: EmbeddingKind::Document,
};
let qry = EmbeddingInput {
text: "스택 자료구조",
kind: EmbeddingKind::Query,
};
// arctic: documents are embedded raw, queries get `query: `.
assert_eq!(prefix_input(arctic_spec(), &doc), "후입선출 자료구조");
assert_eq!(prefix_input(arctic_spec(), &qry), "query: 스택 자료구조");
}
#[test]
fn prefix_handles_empty_text() {
let doc = EmbeddingInput {
text: "",
kind: EmbeddingKind::Document,
};
let qry = EmbeddingInput {
text: "",
kind: EmbeddingKind::Query,
};
assert_eq!(prefix_input(e5_spec(), &doc), "passage: ");
assert_eq!(prefix_input(e5_spec(), &qry), "query: ");
assert_eq!(prefix_input(arctic_spec(), &doc), "");
assert_eq!(prefix_input(arctic_spec(), &qry), "query: ");
}
// ── check_dim ────────────────────────────────────────────────────
#[test]
fn check_dim_passes_for_1024() {
check_dim(1024, 1024).expect("matching dims must pass");
}
#[test]
fn check_dim_rejects_384_vs_1024() {
let err = check_dim(384, 1024).expect_err("dim mismatch must error");
let msg = format!("{err}");
assert!(
msg.contains("384") && msg.contains("1024"),
"error must mention both dims, got: {msg}"
);
}
// ── model guard ──────────────────────────────────────────────────
// A model name not in the registry must fail fast (BEFORE the ~2GB
// download), so we never download one model yet label its vectors with
// another name via model_id() — which would mislabel embedding_version.
#[test]
fn new_rejects_unsupported_model() {
let mut config = kebab_config::Config::defaults();
config.models.embedding.model = "multilingual-e5-small".to_string();
// num_threads defaults to 0, so no global rayon side effect here.
// `.err()` (not `expect_err`) avoids requiring `CandleEmbedder: Debug`
// — it holds a Mutex/Tokenizer and intentionally derives no Debug.
let err = CandleEmbedder::new(&config)
.err()
.expect("unsupported model must error");
let msg = format!("{err:#}");
assert!(
msg.contains("candle provider supports the models"),
"expected model-registry error, got: {msg}"
);
}
}

View File

@@ -1,128 +0,0 @@
//! arctic-embed-l-v2.0 correctness gate (`#[ignore]` — needs the ~2GB candle
//! model + a live Ollama serving `snowflake-arctic-embed2`).
//!
//! This is the load-bearing pooling/prefix check for the arctic integration.
//! The recall measurement that justified adopting arctic (recall@10 130/132)
//! went through Ollama's `snowflake-arctic-embed2`. The candle path
//! re-implements the model (XLM-RoBERTa-large + **CLS** pooling + `query: ` on
//! queries / no prefix on documents). If candle's pooling or prefix is wrong,
//! its vectors silently diverge from the measured route and the 130 number
//! does NOT carry over. This test pins them together: per-sentence cosine
//! between the candle vector and the Ollama vector must be **> 0.99**.
//!
//! `#[ignore]` because it depends on an external Ollama daemon (CI is
//! headless/offline). The leader MUST run it once before merge.
//!
//! ## Manual run
//!
//! 1. Confirm Ollama is reachable and has the model:
//! ```sh
//! curl -s http://192.168.0.47:11434/api/tags # should list snowflake-arctic-embed2
//! ```
//! 2. Run (downloads the ~2GB candle safetensors on first run):
//! ```sh
//! CARGO_TARGET_DIR=/build/out/cargo-target \
//! KEBAB_ARCTIC_OLLAMA_ENDPOINT=http://192.168.0.47:11434 \
//! cargo test -p kebab-embed-candle --test arctic_ollama_parity -- --ignored --nocapture
//! ```
//! The endpoint defaults to `http://192.168.0.47:11434` if the env is unset.
//!
//! Record the printed `ARCTIC_PARITY_SUMMARY cosine_min=...` in
//! `/tmp/arctic-result.md` + `tasks/HOTFIXES.md`.
use kebab_config::Config;
use kebab_core::{Embedder, EmbeddingInput, EmbeddingKind};
use kebab_embed_candle::CandleEmbedder;
use kebab_embed_ollama::OllamaEmbedder;
const DOGFOOD_CONFIG: &str = "/build/dogfood/config.toml";
const DEFAULT_OLLAMA_ENDPOINT: &str = "http://192.168.0.47:11434";
/// Mixed Korean / English + the descriptive-recall shapes arctic was adopted
/// for (synonym / abbreviation / English term). Covers both prefix paths.
const SENTENCES: &[&str] = &[
"스택 자료구조",
"후입선출 방식으로 동작하는 자료구조",
"큐는 선입선출 자료구조이다",
"Rust ownership and the borrow checker",
"소유권과 빌림 검사기는 메모리 안전성을 보장한다",
"SVM 은 support vector machine 의 약자이다",
"정렬 알고리즘의 시간 복잡도",
"The capital of France is Paris.",
];
fn cosine(a: &[f32], b: &[f32]) -> f32 {
let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
let na: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
let nb: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
dot / (na * nb)
}
/// Base config: prefer the canonical dogfood config (for storage/cache roots),
/// fall back to `Config::defaults()` so the test still runs on a bare clone.
fn base_config() -> Config {
Config::load(Some(std::path::Path::new(DOGFOOD_CONFIG))).unwrap_or_else(|_| Config::defaults())
}
#[test]
#[ignore = "needs ~2GB candle model + live Ollama (snowflake-arctic-embed2); run manually before merge"]
fn candle_arctic_matches_ollama_arctic() {
let endpoint = std::env::var("KEBAB_ARCTIC_OLLAMA_ENDPOINT")
.unwrap_or_else(|_| DEFAULT_OLLAMA_ENDPOINT.to_string());
// candle side: the in-process arctic model.
let mut candle_cfg = base_config();
candle_cfg.models.embedding.provider = "candle".to_string();
candle_cfg.models.embedding.model = "snowflake-arctic-embed-l-v2.0".to_string();
candle_cfg.models.embedding.dimensions = 1024;
// Ollama side: the reference route the recall numbers came from.
let mut ollama_cfg = base_config();
ollama_cfg.models.embedding.provider = "ollama".to_string();
ollama_cfg.models.embedding.model = "snowflake-arctic-embed2".to_string();
ollama_cfg.models.embedding.dimensions = 1024;
ollama_cfg.models.embedding.endpoint = Some(endpoint.clone());
let candle = CandleEmbedder::new(&candle_cfg).expect("build candle arctic embedder");
let ollama = OllamaEmbedder::new(&ollama_cfg).expect("build ollama arctic embedder");
// Exercise BOTH prefix paths so a query-side divergence can't hide.
let inputs: Vec<EmbeddingInput> = SENTENCES
.iter()
.flat_map(|s| {
[EmbeddingKind::Document, EmbeddingKind::Query]
.into_iter()
.map(move |kind| EmbeddingInput { text: s, kind })
})
.collect();
let cv = candle.embed(&inputs).expect("candle embed");
let ov = ollama
.embed(&inputs)
.expect("ollama embed (is snowflake-arctic-embed2 pulled @ the endpoint?)");
assert_eq!(cv.len(), ov.len(), "embedding counts must match");
assert_eq!(cv.len(), inputs.len(), "one vector per input");
assert_eq!(candle.dimensions(), 1024);
let mut min_cos = f32::INFINITY;
for (i, inp) in inputs.iter().enumerate() {
assert_eq!(cv[i].len(), 1024, "candle dim");
assert_eq!(ov[i].len(), 1024, "ollama dim");
let c = cosine(&cv[i], &ov[i]);
min_cos = min_cos.min(c);
let kind = match inp.kind {
EmbeddingKind::Document => "doc",
EmbeddingKind::Query => "qry",
};
let preview: String = inp.text.chars().take(36).collect();
println!("[{i:>2}] {kind} cos={c:.6} {preview}");
}
println!("ARCTIC_PARITY_SUMMARY cosine_min={min_cos:.6} endpoint={endpoint}");
assert!(
min_cos > 0.99,
"candle arctic vs Ollama arctic cosine_min={min_cos:.6} ≤ 0.99 — \
pooling/prefix mismatch; the recall=130 measurement will NOT reproduce"
);
}

View File

@@ -1,96 +0,0 @@
//! Parity test (spec §7, `#[ignore]` — needs the ~2GB model + network).
//!
//! Confirms the candle backend reproduces the onnxruntime `FastembedEmbedder`
//! vectors closely enough that no re-index is required (spec D-reindex):
//! per-sentence cosine ≥ 0.9999, and reports the dimension-wise max absolute
//! difference (the number the re-index decision hangs on).
//!
//! Run manually:
//! CARGO_TARGET_DIR=/build/out/cargo-target/target \
//! cargo test -p kebab-embed-candle --release -- --ignored --nocapture
//!
//! Uses the canonical dogfood config so both backends resolve the same model
//! identifiers and cache roots.
use kebab_config::Config;
use kebab_core::{Embedder, EmbeddingInput, EmbeddingKind};
use kebab_embed_candle::CandleEmbedder;
use kebab_embed_local::FastembedEmbedder;
const DOGFOOD_CONFIG: &str = "/build/dogfood/config.toml";
/// Mixed Korean / English parity set (≥ 8 sentences, mirrors the Phase 0 spike).
const SENTENCES: &[&str] = &[
"The quick brown fox jumps over the lazy dog.",
"오늘 날씨가 정말 좋아서 산책을 나가고 싶다.",
"Rust is a systems programming language focused on safety and performance.",
"벡터 검색은 임베딩 사이의 코사인 유사도를 이용한다.",
"Machine learning models require large amounts of training data.",
"한국어와 영어가 섞인 문장도 멀티링구얼 모델은 잘 처리한다.",
"The capital of France is Paris, a city known for its art and culture.",
"이 프로젝트는 로컬 우선 지식 베이스와 검색 증강 생성을 목표로 한다.",
"Database indexing dramatically speeds up query performance.",
"임베딩 모델을 candle 로 옮기면 NUMA 서버에서 안전하게 돌릴 수 있다.",
];
fn cosine(a: &[f32], b: &[f32]) -> f32 {
let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
let na: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
let nb: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
dot / (na * nb)
}
#[test]
#[ignore = "needs ~2GB model + network; run manually for the re-index decision"]
fn candle_matches_fastembed() {
let config = Config::load(Some(std::path::Path::new(DOGFOOD_CONFIG)))
.expect("load dogfood config for parity baseline");
let candle = CandleEmbedder::new(&config).expect("build CandleEmbedder");
let fastembed = FastembedEmbedder::new(&config).expect("build FastembedEmbedder");
// Cover BOTH prefix paths (`passage:` for Document, `query:` for Query) so
// a query-side prefix/pooling divergence can't slip through (reviewer note).
let inputs: Vec<EmbeddingInput> = SENTENCES
.iter()
.flat_map(|s| {
[EmbeddingKind::Document, EmbeddingKind::Query]
.into_iter()
.map(move |kind| EmbeddingInput { text: s, kind })
})
.collect();
let cv = candle.embed(&inputs).expect("candle embed");
let fv = fastembed.embed(&inputs).expect("fastembed embed");
assert_eq!(cv.len(), fv.len(), "embedding counts must match");
assert_eq!(cv.len(), inputs.len(), "one vector per input");
assert_eq!(candle.dimensions(), 1024);
let mut min_cos = f32::INFINITY;
let mut max_abs_diff = 0f32;
for (i, inp) in inputs.iter().enumerate() {
assert_eq!(cv[i].len(), 1024, "candle dim");
assert_eq!(fv[i].len(), 1024, "fastembed dim");
let c = cosine(&cv[i], &fv[i]);
min_cos = min_cos.min(c);
let diff = cv[i]
.iter()
.zip(&fv[i])
.map(|(a, b)| (a - b).abs())
.fold(0f32, f32::max);
max_abs_diff = max_abs_diff.max(diff);
let kind = match inp.kind {
EmbeddingKind::Document => "doc",
EmbeddingKind::Query => "qry",
};
let preview: String = inp.text.chars().take(36).collect();
println!("[{i:>2}] {kind} cos={c:.6} max_abs_diff={diff:.6e} {preview}");
}
println!("PARITY_SUMMARY cosine_min={min_cos:.6} max_abs_diff={max_abs_diff:.6e}");
assert!(
min_cos >= 0.9999,
"candle vs fastembed cosine_min={min_cos:.6} < 0.9999 — investigate before merge"
);
}

View File

@@ -1,32 +0,0 @@
//! Thread-cap test (spec §7). Own integration binary → clean process, so the
//! one-shot global rayon pool is initialized exactly once, by us.
//!
//! Verifies that `apply_thread_cap(4)` sizes the global rayon pool to 4, which
//! is the lever that keeps candle's CPU backend NUMA-safe (vs onnxruntime's
//! hard-coded 48 intra-op threads).
use kebab_embed_candle::apply_thread_cap;
#[test]
fn thread_cap_sizes_global_rayon_pool() {
// Must run before any other rayon use in this process. As the only test in
// this binary that touches rayon, that holds.
let applied = apply_thread_cap(4);
assert!(applied, "first build_global call should succeed");
assert_eq!(
rayon::current_num_threads(),
4,
"global rayon pool must be capped at the requested 4 threads"
);
// A second cap attempt is a no-op (pool already built), not a panic.
assert!(
!apply_thread_cap(8),
"second build_global must report not-applied"
);
assert_eq!(
rayon::current_num_threads(),
4,
"thread count must stay at the first cap"
);
}

View File

@@ -4,11 +4,10 @@
//!
//! ## Why this exists
//!
//! The candle backend ([`kebab-embed-candle`]) runs arctic-embed-l-v2.0
//! in-process (pure Rust, NUMA-safe). This crate is the **fallback** path:
//! it offloads embedding to a local/remote Ollama daemon (`snowflake-arctic-embed2`),
//! which is exactly the route the recall measurements used — so it reproduces
//! the measured numbers (recall@10 130/132) byte-for-route. Opt-in via
//! This crate offloads embedding to a local/remote Ollama daemon
//! (`snowflake-arctic-embed2`), which is exactly the route the recall
//! measurements used — so it reproduces the measured numbers (recall@10
//! 130/132) byte-for-route. Opt-in via
//! `config.models.embedding.provider = "ollama"`.
//!
//! ## Wire shape
@@ -64,10 +63,9 @@ const REQUEST_TIMEOUT_SECS: u64 = 300;
/// Resolve the (query_prefix, doc_prefix) for an Ollama embedding model tag.
///
/// Mirrors `kebab-embed-candle`'s `MODEL_REGISTRY`, but keyed on the **Ollama
/// model tag** (which differs from the HF id — e.g. `snowflake-arctic-embed2`
/// vs `Snowflake/snowflake-arctic-embed-l-v2.0`). Kept here rather than shared
/// so this crate does not depend on the candle backend.
/// Resolve the (query_prefix, doc_prefix) for an Ollama embedding model tag,
/// keyed on the **Ollama model tag** (which differs from the HF id — e.g.
/// `snowflake-arctic-embed2` vs `Snowflake/snowflake-arctic-embed-l-v2.0`).
///
/// An unrecognized model gets no prefix (`("", "")`): many embedding models
/// are not instruction-tuned, so embedding the raw text is the correct default

View File

@@ -16,7 +16,7 @@ Cargo workspace, 함수 호출 기반 모듈러 모놀리스. UI binary (`kebab-
| metadata | SQLite + FTS5 (lexical search + v0.20.1 한국어 형태소 tokenizer via lindera-ko-dic) |
| vector | LanceDB (embedded, model 별 분리 table) |
| Markdown parser | `pulldown-cmark`. frontmatter 에 title 없으면 첫 H1 → H2 → 첫 paragraph 80 자 → 파일명 순으로 자동 채움 (`parser_version = md-frontmatter-v2`, 기존 doc 도 다음 ingest 에서 갱신) |
| embedding | `fastembed-rs` (`multilingual-e5-large`, 1024d, v0.18.0부터 default 업그레이드). opt-in 대안: candle (e5 또는 `snowflake-arctic-embed-l-v2.0`) / Ollama `/api/embed`. arctic = 설명형 query recall 보강 (v0.26.0, 아래 결정표) |
| embedding | `fastembed-rs` (`multilingual-e5-large`, 1024d, v0.18.0부터 default 업그레이드). opt-in 대안: Ollama `/api/embed` (`snowflake-arctic-embed-l-v2.0` 등). arctic = 설명형 query recall 보강 (v0.26.0, 아래 결정표) |
| 한국어 형태소분석 | `lindera-ko-dic` (FTS5 외부 tokenizer, v0.20.1) — 2자 이상 한국어 query 지원 |
| LLM | Ollama HTTP (default `gemma4:e4b` ─ OCR / caption 와 family 통일. 사용자가 더 큰 variant `gemma4:26b` 등으로 override 가능) |
| 음성 ASR | `whisper.cpp` (via `whisper-rs`) — P8 보류, 시스템 dep brainstorm 후 |
@@ -68,7 +68,6 @@ flowchart TB
subgraph Adapters ["traits + adapters"]
embed["kebab-embed<br/>(trait)"]
embedlocal["kebab-embed-local<br/>(fastembed, default)"]
embedcandle["kebab-embed-candle<br/>(candle, e5+arctic, NUMA-safe opt-in)"]
embedollama["kebab-embed-ollama<br/>(Ollama /api/embed, opt-in)"]
llm["kebab-llm<br/>(trait)"]
llmlocal["kebab-llm-local<br/>(Ollama)"]
@@ -95,7 +94,6 @@ flowchart TB
app --> sqlite
app --> vector
app --> embedlocal
app --> embedcandle
app --> embedollama
app --> llmlocal
app --> search
@@ -109,8 +107,6 @@ flowchart TB
paud --> core
pcode --> core
embedlocal --> embed
embedcandle --> core
embedcandle --> config
embedollama --> core
embedollama --> config
llmlocal --> llm
@@ -145,16 +141,13 @@ UI → store/llm/parse 직접 의존 금지. 모든 user-facing 진입은 `kebab
| provider | 모델 | pooling / prefix | 위치 | 언제 |
|---|---|---|---|---|
| `fastembed` (기본) | `multilingual-e5-large` | mean / `query:`·`passage:` | in-process (onnxruntime) | 기본. 단일 소켓 호스트 |
| `candle` | e5 또는 `snowflake-arctic-embed-l-v2.0` | 모델별 (e5=mean, arctic=CLS) / arctic=`query:`·무접두어 | in-process (pure Rust) | NUMA 서버 (onnxruntime 48-스레드 double-free 회피), Apple Silicon Metal GPU |
| `ollama` | `snowflake-arctic-embed2` 등 | 모델 태그로 추론 / arctic=`query:`·무접두어 | 원격 HTTP (`/api/embed`) | candle 폴백, 측정에 쓴 경로 그대로 재현 |
| `fastembed` (기본) | `multilingual-e5-large` | mean / `query:`·`passage:` | in-process (onnxruntime) | 기본. 모든 호스트 |
| `ollama` | `snowflake-arctic-embed2` 등 | 모델 태그로 추론 / arctic=`query:`·무접두어 | 원격 HTTP (`/api/embed`) | GPU 서버 위임, 측정에 쓴 경로 그대로 재현 |
**arctic-embed-l-v2.0 채택 근거**: 별칭(doc-side expansion) 제거(v0.25.0) 후 설명형
query 의 recall 보강책. 측정(`/build/dogfood/logs/2026-06-03-method-measurements.md`)에서
arctic = recall@10 130/132 (e5 대비 +7, 색인 1회·per-query 0·LLM 0, 용어 무손실).
candle 이 주 백엔드(in-process, NUMA 안전), Ollama 가 폴백(측정 경로 재현). 두 경로의
pooling/prefix 정확성은 `kebab-embed-candle/tests/arctic_ollama_parity.rs`
(candle arctic vs Ollama arctic 코사인>0.99, `#[ignore]`) 로 고정. e5 → arctic 전환은
Ollama 백엔드(`provider = "ollama"`)로 arctic 모델 사용. e5 → arctic 전환은
`embedding_version` cascade (모델별 벡터 상이) → 재색인 필요. 기본값 e5 유지라 기존
사용자 무영향. 자세한 내용: [tasks/HOTFIXES.md](../tasks/HOTFIXES.md) 2026-06-03 arctic entry.
@@ -206,8 +199,7 @@ kebab/
│ ├── kebab-store-sqlite/ # SQLite + FTS5 (V001/V002/V003) (P1-6, P2-1, P3-3). src/derivation_cache.rs = derivation_cache 테이블 저장소 (V012, v0.21.0)
│ ├── kebab-search/ # Lexical + Vector + Hybrid retriever (P2-2, P3-4)
│ ├── kebab-embed/ kebab-embed-local/ # Embedder trait + fastembed adapter (P3-1, P3-2)
│ ├── kebab-embed-candle/ # candle (pure-Rust) Embedder, 모델 레지스트리(e5 mean + arctic CLS), NUMA-safe opt-in provider=candle (Track 1, v0.22.0; arctic v0.26.0)
│ ├── kebab-embed-ollama/ # Ollama /api/embed Embedder, opt-in provider=ollama (arctic 폴백 경로, v0.26.0)
│ ├── kebab-embed-ollama/ # Ollama /api/embed Embedder, opt-in provider=ollama (arctic 경로, v0.26.0)
│ ├── kebab-store-vector/ # LanceDB VectorStore (P3-3, P7-3 follow-up)
│ ├── kebab-llm/ kebab-llm-local/ # LanguageModel trait + Ollama adapter (P4-1, P4-2)
│ ├── kebab-rag/ # RAG pipeline (P4-3)