refactor(embed): candle provider/crate 제거 — fastembed+ollama로 충분
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Mc6W1fgsrbFKTsqA6P8La
This commit is contained in:
777
Cargo.lock
generated
777
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
@@ -11,7 +11,6 @@ members = [
|
||||
"crates/kebab-search",
|
||||
"crates/kebab-embed",
|
||||
"crates/kebab-embed-local",
|
||||
"crates/kebab-embed-candle",
|
||||
"crates/kebab-embed-ollama",
|
||||
"crates/kebab-llm",
|
||||
"crates/kebab-llm-local",
|
||||
|
||||
@@ -30,7 +30,7 @@ P0~P5 직렬. P6~P9 P5 이후 병렬 가능.
|
||||
|
||||
## 머지 후 발견된 버그 / 결정 (요약)
|
||||
|
||||
- **candle 임베딩 백엔드 다변화** (2026-06-01, Track 1, v0.22.0): `provider = "candle"` opt-in 추가 — 같은 `multilingual-e5-large` 모델을 순수 Rust(candle)로 돌려 듀얼소켓 NUMA 서버의 onnxruntime 48-스레드 double-free 를 회피. `[models.embedding].num_threads`(+env `KEBAB_EMBED_THREADS`)로 CPU 스레드 캡. fastembed default 동작·벡터 불변, `embedding_version` 유지(재색인 0). Phase 0 스파이크 패리티 cosine 1.000000. 상세 HOTFIXES 동일 일자.
|
||||
- **candle 임베딩 백엔드 제거** (2026-06-24, spine-phase0): `provider = "candle"` 제거 — fastembed(기본) + ollama 두 경로로 충분, candle 은 미사용. `num_threads` 필드는 TOML 하위호환 위해 유지(레거시, 무시). 상세: spine-cuts 계획.
|
||||
- **config 마이그레이션** (2026-05-31, PR #198): `kebab config migrate` 추가 — 기존 config.toml 에 빠진 섹션을 주석과 함께 채우고 deprecated 정리(멱등·`.bak`·dry-run, 값/주석 보존). `schema_version` 1→2, `init` 도 섹션 주석 포함, doctor 에 `config_migration` 체크. 상세 HOTFIXES 동일 일자.
|
||||
|
||||
머지 후 발견된 모든 deviation / hotfix 의 dated 로그는 [tasks/HOTFIXES.md](tasks/HOTFIXES.md). 본 요약은 \"누군가가 인수받을 때 알아두면 시간을 많이 절약하는\" 항목만:
|
||||
@@ -40,7 +40,7 @@ P0~P5 직렬. P6~P9 P5 이후 병렬 가능.
|
||||
- **2026-06-04 PP-OCRv5 ONNX Rust 네이티브 OCR** — v0.27.0. `[image.ocr] engine = "paddle-onnx"` 로 PP-OCRv5(검출+인식) ONNX 를 in-process(`ort` =2.0.0-rc.9) 실행 — Python 런타임/원격 호출 없이 큰 페이지 CPU <4초(Ollama vision ~50초 대비). default 는 여전히 `"ollama-vision"`. 후처리(min-area rect/unclip)는 pure-Rust. **함정**: unclip 은 corner 를 centroid 에서 방사 확장하면 안 되고 edge 별 polygon offset 이어야 함(방사 확장 시 wide/short 텍스트 박스 높이가 안 커져 글자 윗부분 잘림 → ㄷ→ㄴ, e2e CER 0.26). 수정 후 CER 0.005. 모델 ONNX 는 `crates/kebab-parse-image/assets/paddleocr-onnx/`(LFS). 자세한 내용: `tasks/HOTFIXES.md` (2026-06-04 PP-OCRv5 ONNX), spec/plan `docs/superpowers/{specs,plans}/2026-06-04-rust-native-ocr-*.md`.
|
||||
- **2026-06-03 ingest 설정 변경 자동 재색인** — v0.26.2. ingest 산출에 영향 주는 설정(청킹/이미지 OCR·caption/pdf.ocr/`[ingest.code]`)을 변경하면 `--force-reingest` 없이 영향 자산만 자동 재색인. 그 설정들의 결정적 서명(`ingest_config_signature`)을 effective parser_version(skip 비교 + 저장 doc 필드 양쪽)에 폴딩 → 다음 ingest 비교가 mismatch. 비산출 설정(search/rag/ui/log + max_pixels/languages/timeout)은 제외(과도 무효화 회피), doc_id 는 base 로 안정 유지. **업그레이드 후 첫 ingest 는 전 자산 1회 재색인**(저장된 상수 parser_version ≠ 새 composite; embedding 은 V012 캐시 히트). 결과 포맷·CLI·wire 불변(내부 skip 판정 정정). 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 ingest 설정 변경 자동 재색인), spec/plan `docs/superpowers/{specs,plans}/2026-06-03-*invalidation*.md`.
|
||||
- **2026-06-03 ingest 진행 로그 개선** — v0.26.1. 이미지/PDF + OCR/caption on 볼트 ingest 가 "멈춘 듯" 보이던 문제 해소: TTY 진행바에 현재 파일명 + 느린 phase(ocr/caption/embed)+모델명 + 경과초 `(Ns)` heartbeat, 종료 시 최장 소요 파일 top-5 요약. 신규 wire `asset_phase{idx,total,phase,model}` + `asset_timings.ocr_ms`/`caption_ms`(additive, `ingest_progress.v1` 유지, serde default 0). 이미지·PDF 경로도 `asset_timings` emit(이전 markdown 만). 기본 동작 불변. 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 ingest 진행 로그), spec/plan `docs/superpowers/{specs,plans}/2026-06-03-ingest-log-improve-*.md`.
|
||||
- **2026-06-03 arctic-embed-l-v2.0 임베더 통합** — v0.26.0. 별칭 제거 후 설명형 query recall 보강(측정 recall@10 130/132, e5 +7). `kebab-embed-candle` 모델 레지스트리화(e5 mean + `snowflake-arctic-embed-l-v2.0` CLS, 모델별 pooling/prefix) + 신규 `kebab-embed-ollama`(`provider="ollama"`, `/api/embed`). config `endpoint: Option<String>` 추가. 기본 e5 유지(opt-in), arctic 전환은 embedding_version cascade → 재색인. candle↔Ollama cosine>0.99 게이트로 pooling/prefix 정확성 고정(`#[ignore]`). 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 arctic), spec `docs/superpowers/specs/2026-06-03-arctic-embedder-spec.md`.
|
||||
- **2026-06-03 arctic-embed-l-v2.0 임베더 통합** — v0.26.0. 별칭 제거 후 설명형 query recall 보강(측정 recall@10 130/132, e5 +7). 신규 `kebab-embed-ollama`(`provider="ollama"`, `/api/embed`). config `endpoint: Option<String>` 추가. 기본 e5 유지(opt-in), arctic 전환은 embedding_version cascade → 재색인. 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03 arctic), spec `docs/superpowers/specs/2026-06-03-arctic-embedder-spec.md`.
|
||||
- **2026-06-03 doc-side expansion(별칭) 기능 완전 제거** — v0.25.0. 아래 2026-05-31 항목의 색인-시 청크당 LLM 별칭 생성 + 별칭 검색 채널을 **전부 제거**(ROI 음수: cross-lingual 은 e5-large 단독으로 충분, 기여는 설명형 +2 그룹뿐인데 대가가 청크당 색인-시 LLM). `Chunk.aliases`/`expansion.rs`/`IngestExpansionCfg`/alias lexical arm/`expansion_progress` wire kind 제거, 신규 마이그레이션 **V013** 이 `chunk_aliases_fts`+`chunks.aliases` DROP. 별칭 default-off 였어 사용자 체감 0, 기존 KB 도 재색인 불요(잔존 별칭 벡터는 `strip_alias_suffix` graceful 매핑/`reset` 정리). `AssetTimings.expansion_ms` 는 wire 호환 위해 값 0 으로 유지. 자세한 내용: `tasks/HOTFIXES.md` (2026-06-03), spec `docs/superpowers/specs/2026-06-03-remove-doc-expansion-spec.md`.
|
||||
- **2026-05-31 Phase 2 doc-side expansion 별칭(개별 dense 벡터) + 파생물 캐시(V012)** — v0.21.0 cut. 색인 시 LLM 이 청크별 별칭("같은 의미 다른 표현")을 생성, 줄별 **개별 dense 벡터**(sentinel `{chunk}#alias#N`)로 색인 (묶음 1벡터는 평균화 희석으로 회귀 → 폐기) + boilerplate 청크 skip. `[ingest.expansion]` default off. 측정(나무위키 ~1000 문서 CS corpus): 변형 일관성 14/18 → **16/18**, spread 0.222→0.111, 대조군 false-positive 별칭 무죄. 비용 병목(별칭 18문서 2.5h)은 **파생물 캐시(V012, 청크 내용 해시 키)**로 해소 — 정답 3개 cold 1879s → warm 13s **≈ 145배**, embedding+별칭 LLM 캐싱, version_key cascade 정합. search/ask 가 `kebab.sqlite`+`lancedb` 만으로 동작 → 외부 서버 색인 후 DB 만 복사하는 이식 워크플로 가능. **결정/known limitation**: grounded/refusal 판정이 부분 인용을 grounded 로 오분류(정직한 거부가 false-positive 로 집계) — 별도 개선 후보. stack·svm 설명형 2개 잔존. 자세한 내용: `tasks/HOTFIXES.md` (2026-05-31), 측정: `docs/superpowers/handoffs/2026-05-31-namu-wiki-alias-cache-study.md`.
|
||||
- **2026-05-29 v0.20.2 dogfood findings + 검색 품질 baseline** — 8-finding 라운드 완료. (1) Ask 응답언어: rag-v3 default (질문 언어 = 답변 언어). (2) eval `--config` facade 패치 로 dogfood KB 직접 eval 가능. (3) 검색 품질 baseline — hybrid hit@3=1.0 / MRR=0.833, lexical hit@3=1.0 / MRR=0.7 (golden 10 query). **O-2 known limitation**: 소형 모델(gemma4:e4b) refusal 메시지의 query 언어 불일치 가능 — 판정은 정상, 표시 문구만 해당. 자세한 내용: `tasks/HOTFIXES.md` (2026-05-29).
|
||||
|
||||
42
README.md
42
README.md
@@ -47,7 +47,7 @@ embedding 벡터를 청크 **내용 해시** 로 캐싱한다 (`derivation_cache
|
||||
|
||||
### 외부 계산 + 로컬 검색 워크플로
|
||||
|
||||
search/ask 는 원본 파일 없이 KB 산출물만으로 동작한다 (청크 본문이 SQLite 에 저장되고 문서 경로는 상대경로로 기록됨). 비싼 색인(임베딩·OCR)을 성능 좋은 머신에서 수행한 뒤(예: Apple Silicon 맥에서 candle Metal GPU), **두 산출물만** 다른 머신(예: NUMA 서버)으로 복사하면 그대로 검색·질문할 수 있다.
|
||||
search/ask 는 원본 파일 없이 KB 산출물만으로 동작한다 (청크 본문이 SQLite 에 저장되고 문서 경로는 상대경로로 기록됨). 비싼 색인(임베딩·OCR)을 성능 좋은 머신에서 수행한 뒤, **두 산출물만** 다른 머신으로 복사하면 그대로 검색·질문할 수 있다.
|
||||
|
||||
**무엇을 복사하나 — `[storage]` 에서 정의된 두 경로:**
|
||||
|
||||
@@ -64,7 +64,7 @@ rsync -a <src-data_dir>/kebab.sqlite user@server:<dst-data_dir>/
|
||||
rsync -a <src-data_dir>/lancedb/ user@server:<dst-data_dir>/lancedb/
|
||||
```
|
||||
|
||||
조건: **양쪽 동일 `kebab` 버전 + 동일 임베딩 모델/차원** (`[models.embedding].model`·`dimensions`). provider 는 달라도 됨 (예: 맥 `candle`/Metal ↔ 서버 `candle`/CPU 또는 `fastembed` — 같은 모델이면 벡터 호환). 복사는 반드시 ingest 가 돌지 않을 때.
|
||||
조건: **양쪽 동일 `kebab` 버전 + 동일 임베딩 모델/차원** (`[models.embedding].model`·`dimensions`). provider 는 달라도 됨 (예: `fastembed` ↔ `ollama` — 같은 모델이면 벡터 호환). 복사는 반드시 ingest 가 돌지 않을 때.
|
||||
|
||||
### 멀티미디어 색인
|
||||
|
||||
@@ -123,23 +123,16 @@ root = "~/KnowledgeBase" # 색인할 폴더. 절대 / tilde / env / 상대 경
|
||||
# trust_level = "secondary" # 낮은 신뢰 출처 — `--trust-min primary` 로 배제 가능.
|
||||
|
||||
[models.embedding]
|
||||
provider = "fastembed" # "fastembed"(기본, onnxruntime) / "candle"(순수 Rust)
|
||||
# / "ollama"(원격 HTTP) / "none"(lexical-only).
|
||||
# candle 는 같은 모델·같은 벡터를 순수 Rust 로 돌려
|
||||
# NUMA 서버의 onnxruntime 48-스레드 double-free 를 피하는
|
||||
# opt-in 백엔드 (e5 는 재색인 불필요).
|
||||
provider = "fastembed" # "fastembed"(기본, onnxruntime) / "ollama"(원격 HTTP)
|
||||
# / "none"(lexical-only).
|
||||
model = "multilingual-e5-large" # 다국어 sentence embedding (1024-dim).
|
||||
# 첫 ingest 시 ONNX (~1.3GB) 자동 다운로드.
|
||||
# candle provider 는 safetensors (~2GB) 다운로드.
|
||||
# candle/ollama 는 "snowflake-arctic-embed-l-v2.0"
|
||||
# ollama 는 "snowflake-arctic-embed-l-v2.0"
|
||||
# (설명형 query 의 recall 보강) 도 지원 — 아래 참고.
|
||||
dimensions = 1024 # config 와 LanceDB stored dim 불일치 시 검색 0건.
|
||||
num_threads = 0 # candle 전용 CPU 스레드 캡 (0=auto=#cores).
|
||||
# env KEBAB_EMBED_THREADS 가 우선. NUMA 노드 바인딩은
|
||||
# numactl 과 조합. fastembed provider 는 무시.
|
||||
# endpoint = "http://127.0.0.1:11434" # provider="ollama" 전용 HTTP endpoint.
|
||||
# 생략 시 [models.llm].endpoint 로 폴백.
|
||||
# fastembed/candle provider 는 무시.
|
||||
# fastembed provider 는 무시.
|
||||
```
|
||||
|
||||
**arctic-embed-l-v2.0 (설명형 query recall 보강)**: 기본 e5-large 대신
|
||||
@@ -147,13 +140,7 @@ Snowflake `arctic-embed-l-v2.0` 임베더를 쓸 수 있다 (1024-dim, opt-in).
|
||||
설명형/약어/영문 용어 query 의 recall@10 이 e5 대비 향상됐다. 두 경로:
|
||||
|
||||
```toml
|
||||
# (A) candle 백엔드 — 순수 Rust, in-process (NUMA 안전, Metal GPU 가능):
|
||||
[models.embedding]
|
||||
provider = "candle"
|
||||
model = "snowflake-arctic-embed-l-v2.0" # CLS pooling, query 에 "query: " 접두어
|
||||
# (문서는 무접두어). safetensors ~2GB 다운로드.
|
||||
|
||||
# (B) ollama 백엔드 — 원격/로컬 Ollama 데몬에 위임 (POST /api/embed):
|
||||
# ollama 백엔드 — 원격/로컬 Ollama 데몬에 위임 (POST /api/embed):
|
||||
[models.embedding]
|
||||
provider = "ollama"
|
||||
model = "snowflake-arctic-embed2" # Ollama 모델 태그 (ollama pull 필요)
|
||||
@@ -164,21 +151,6 @@ endpoint = "http://127.0.0.1:11434" # 생략 시 [models.llm].endpoint
|
||||
> 벡터도 다름). 기존 e5 KB 와 혼용 불가 — 전환 시 **재색인** 필요 (`kebab reset`
|
||||
> 후 재 ingest). 기본값은 e5 라 기존 사용자는 영향 없음.
|
||||
|
||||
**Apple Silicon GPU 가속 (candle / macOS)**: M-시리즈 맥에서 candle 임베딩을
|
||||
GPU(Metal)로 돌리면 CPU 대비 대용량 ingest 가 크게 빨라진다. 빌드 또는 설치 시
|
||||
`embed_metal` feature 를 켠다:
|
||||
|
||||
```bash
|
||||
# 빌드만:
|
||||
cargo build --release --features embed_metal
|
||||
# 전역 설치 (~/.cargo/bin/kebab):
|
||||
cargo install --path crates/kebab-cli --features embed_metal --locked
|
||||
```
|
||||
|
||||
벡터는 CPU candle 과 동일 모델이라 호환되므로, 맥에서 GPU 로 색인한
|
||||
`kebab.sqlite` + `lancedb/` 를 그대로 Linux 서버(CPU candle)로 복사해 질의할 수
|
||||
있다. 색인 로그에 `candle device = Metal (GPU)` 가 보이면 GPU 사용 중. metal
|
||||
feature 는 macOS 전용 (Linux/서버는 기본 CPU 빌드).
|
||||
|
||||
```toml
|
||||
|
||||
|
||||
@@ -18,7 +18,6 @@ kebab-store-vector = { path = "../kebab-store-vector" }
|
||||
kebab-search = { path = "../kebab-search" }
|
||||
kebab-embed = { path = "../kebab-embed" }
|
||||
kebab-embed-local = { path = "../kebab-embed-local" }
|
||||
kebab-embed-candle = { path = "../kebab-embed-candle" }
|
||||
kebab-embed-ollama = { path = "../kebab-embed-ollama" }
|
||||
kebab-llm = { path = "../kebab-llm" }
|
||||
kebab-llm-local = { path = "../kebab-llm-local" }
|
||||
@@ -95,8 +94,6 @@ reqwest = { version = "0.12", default-features = false, features = ["blocki
|
||||
# disable path 없음; 이 feature 는 spec §6.3 명시를 honor 하는 role 만.
|
||||
default = ["fts_korean_morphological"]
|
||||
fts_korean_morphological = []
|
||||
# opt-in (macOS): candle embedder runs on the Apple Silicon GPU. See kebab-embed-candle.
|
||||
embed_metal = ["kebab-embed-candle/metal"]
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
||||
@@ -41,7 +41,6 @@ use kebab_core::{
|
||||
Answer, DocumentStore, Embedder, ExtractContext, Extractor, IndexVersion, LanguageModel,
|
||||
MediaType, Retriever, SearchHit, SearchMode, SearchOpts, SearchQuery, VectorStore,
|
||||
};
|
||||
use kebab_embed_candle::CandleEmbedder;
|
||||
use kebab_embed_local::FastembedEmbedder;
|
||||
use kebab_embed_ollama::OllamaEmbedder;
|
||||
use kebab_llm_local::OllamaLanguageModel;
|
||||
@@ -607,28 +606,23 @@ impl App {
|
||||
if let Some(e) = self.embedder.get() {
|
||||
return Ok(Some(e.clone()));
|
||||
}
|
||||
// Provider branch (Track 1 spec §3 + arctic-embedder spec). The
|
||||
// `embeddings_disabled()` check above already handled `"none"`; here we
|
||||
// route the live providers. `fastembed`/`onnx`/(empty) keep the default
|
||||
// onnxruntime path (vectors unchanged — `embedding_version` is
|
||||
// preserved); `candle` selects the pure-Rust NUMA-safe backend (e5 or
|
||||
// arctic via its model registry); `ollama` offloads to a remote
|
||||
// `/api/embed` daemon.
|
||||
// Provider branch (arctic-embedder spec). The `embeddings_disabled()`
|
||||
// check above already handled `"none"`; here we route the live
|
||||
// providers. `fastembed`/`onnx`/(empty) keep the default onnxruntime
|
||||
// path (vectors unchanged — `embedding_version` is preserved); `ollama`
|
||||
// offloads to a remote `/api/embed` daemon.
|
||||
let provider = self.config.models.embedding.provider.as_str();
|
||||
let emb: Arc<dyn Embedder + Send + Sync> = match provider {
|
||||
"fastembed" | "onnx" | "" => Arc::new(
|
||||
FastembedEmbedder::new(&self.config).context("kb-app: load FastembedEmbedder")?,
|
||||
),
|
||||
"candle" => Arc::new(
|
||||
CandleEmbedder::new(&self.config).context("kb-app: load CandleEmbedder")?,
|
||||
),
|
||||
"ollama" => Arc::new(
|
||||
OllamaEmbedder::new(&self.config).context("kb-app: load OllamaEmbedder")?,
|
||||
),
|
||||
other => {
|
||||
return Err(anyhow!(
|
||||
"kb-app: unknown embedding provider {other:?}; expected one of \
|
||||
`fastembed` (default), `candle`, `ollama`, or `none` (lexical-only)"
|
||||
`fastembed` (default), `ollama`, or `none` (lexical-only)"
|
||||
));
|
||||
}
|
||||
};
|
||||
|
||||
@@ -51,10 +51,5 @@ tempfile = { workspace = true }
|
||||
rusqlite = { workspace = true }
|
||||
time = { workspace = true }
|
||||
|
||||
[features]
|
||||
# opt-in (macOS): build the `kebab` binary with candle on the Apple Silicon GPU.
|
||||
# cargo build --release --features embed_metal
|
||||
embed_metal = ["kebab-app/embed_metal"]
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
||||
@@ -654,14 +654,9 @@ fn run(cli: &Cli) -> anyhow::Result<()> {
|
||||
let mode = progress::ProgressMode::from_flags(cli.json, cli.quiet, plain_env);
|
||||
|
||||
// Surface the active embedding backend/device on the terminal so the
|
||||
// user sees it without grepping kb.log (the per-device tracing line
|
||||
// only lands in the log file at --verbose). Suppressed under
|
||||
// --json/--quiet. The Metal note reflects the build (`embed_metal`);
|
||||
// the confirmed runtime device is in kb.log (`candle device = ...`).
|
||||
// user sees it without grepping kb.log. Suppressed under --json/--quiet.
|
||||
if !cli.json && !cli.quiet {
|
||||
let backend = match cfg.models.embedding.provider.as_str() {
|
||||
"candle" if cfg!(feature = "embed_metal") => "candle (Metal/GPU 빌드)",
|
||||
"candle" => "candle (CPU, 순수 Rust)",
|
||||
"fastembed" | "onnx" | "" => "fastembed (onnxruntime)",
|
||||
"none" => "비활성 (lexical-only)",
|
||||
other => other,
|
||||
|
||||
@@ -255,26 +255,24 @@ impl NliCfg {
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct EmbeddingModelCfg {
|
||||
/// `fastembed` (default, onnxruntime), `candle` (pure-Rust, NUMA-safe),
|
||||
/// or `ollama` (remote HTTP embedding endpoint). `none` disables
|
||||
/// embeddings (lexical-only). Unknown values error at embedder
|
||||
/// construction.
|
||||
/// `fastembed` (default, onnxruntime) or `ollama` (remote HTTP embedding
|
||||
/// endpoint). `none` disables embeddings (lexical-only). Unknown values
|
||||
/// error at embedder construction.
|
||||
pub provider: String,
|
||||
pub model: String,
|
||||
pub version: String,
|
||||
pub dimensions: usize,
|
||||
pub batch_size: usize,
|
||||
/// Cap on the CPU worker threads the `candle` provider spins up
|
||||
/// (sizes the global rayon pool; env `KEBAB_EMBED_THREADS` overrides).
|
||||
/// `0` = auto (rayon default = #cores). Lever to sidestep the
|
||||
/// onnxruntime 48-thread NUMA double-free; ignored by the `fastembed`
|
||||
/// provider. Defaulted on load so pre-0.22 config files still parse.
|
||||
/// Legacy field — previously used by the removed `candle` provider to cap
|
||||
/// CPU worker threads. Retained for backward-compatible TOML parsing
|
||||
/// (`num_threads = 0` in old config files must not error). Ignored by all
|
||||
/// current providers.
|
||||
#[serde(default)]
|
||||
pub num_threads: u32,
|
||||
/// HTTP endpoint for the `ollama` embedding provider (e.g.
|
||||
/// `"http://127.0.0.1:11434"`). `None` (or a missing key in TOML) means
|
||||
/// "fall back to `models.llm.endpoint`" — same convention as the OCR /
|
||||
/// vision endpoints. Ignored by the `fastembed` / `candle` providers.
|
||||
/// vision endpoints. Ignored by the `fastembed` provider.
|
||||
/// Defaulted on load so pre-0.26 config files still parse.
|
||||
#[serde(default)]
|
||||
pub endpoint: Option<String>,
|
||||
|
||||
@@ -99,9 +99,9 @@ fn key_comment(path: &str) -> Option<&'static str> {
|
||||
"workspace.root" => "색인 루트. 절대/~/${VAR}/상대(=이 파일 기준).",
|
||||
"workspace.exclude" => "denylist glob.",
|
||||
"storage.copy_threshold_mb" => "이 크기(MB) 초과 파일은 사본 대신 참조.",
|
||||
"models.embedding.provider" => "fastembed | candle | ollama | none.",
|
||||
"models.embedding.provider" => "fastembed | ollama | none.",
|
||||
"models.embedding.dimensions" => "모델 출력 차원. 틀리면 검색 0건.",
|
||||
"models.embedding.num_threads" => "candle 전용 CPU 스레드 cap(0=auto).",
|
||||
"models.embedding.num_threads" => "레거시 필드 (deprecated, 무시됨).",
|
||||
"models.embedding.endpoint" => "ollama provider 시 HTTP. 비우면 llm.endpoint fallback.",
|
||||
"models.llm.request_timeout_secs" => "단일 HTTP 상한. 0=즉시실패(비활성화 아님).",
|
||||
"ingest.max_parallel_extractors" => "동시 extractor 수.",
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
[package]
|
||||
name = "kebab-embed-candle"
|
||||
version = { workspace = true }
|
||||
edition = { workspace = true }
|
||||
rust-version = { workspace = true }
|
||||
license = { workspace = true }
|
||||
repository = { workspace = true }
|
||||
description = "Pure-Rust candle adapter implementing kb_core::Embedder (multilingual-e5-large, NUMA-safe thread cap)"
|
||||
|
||||
[dependencies]
|
||||
kebab-core = { path = "../kebab-core" }
|
||||
kebab-config = { path = "../kebab-config" }
|
||||
# candle stack — pinned to the workspace-locked crates.io release (0.10.x),
|
||||
# same versions the Phase 0 spike compiled so build artifacts are reused.
|
||||
candle-core = "0.10.2"
|
||||
candle-nn = "0.10.2"
|
||||
candle-transformers = "0.10.2"
|
||||
tokenizers = "0.21"
|
||||
hf-hub = { version = "0.4", features = ["ureq"] }
|
||||
serde_json = { workspace = true }
|
||||
# Thread cap: a one-shot global rayon pool sizes candle's CPU threads
|
||||
# (the Phase 0 spike proved RAYON_NUM_THREADS caps candle), so a NUMA host
|
||||
# can keep onnxruntime's hard-coded 48-intra-op heap corruption at bay.
|
||||
rayon = "1"
|
||||
anyhow = { workspace = true }
|
||||
tracing = { workspace = true }
|
||||
|
||||
[features]
|
||||
# opt-in: run candle on the Apple Silicon GPU (Metal). macOS-only — the build
|
||||
# enables candle's metal backend and `select_device()` picks Metal (CPU fallback
|
||||
# on failure). Lets an M-series Mac ingest e5-large on GPU (10×+ vs CPU); the
|
||||
# resulting vectors are cross-compatible with the CPU path (same model), so the
|
||||
# Linux server can serve queries on CPU candle.
|
||||
metal = ["candle-core/metal", "candle-nn/metal", "candle-transformers/metal"]
|
||||
|
||||
[dev-dependencies]
|
||||
# Integration-test binaries can only see the library's public API + these,
|
||||
# not the library's own (non-dev) dependencies — so rayon/kebab-config/kebab-core
|
||||
# are repeated here for tests/parity.rs and tests/thread_cap.rs.
|
||||
kebab-embed-local = { path = "../kebab-embed-local" }
|
||||
# arctic↔Ollama parity test drives the real Ollama adapter for the reference
|
||||
# vectors (tests/arctic_ollama_parity.rs, `#[ignore]` — live Ollama).
|
||||
kebab-embed-ollama = { path = "../kebab-embed-ollama" }
|
||||
kebab-config = { path = "../kebab-config" }
|
||||
kebab-core = { path = "../kebab-core" }
|
||||
rayon = "1"
|
||||
tempfile = { workspace = true }
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
@@ -1,619 +0,0 @@
|
||||
//! `kebab-embed-candle` — [`CandleEmbedder`], a pure-Rust (candle)
|
||||
//! implementation of [`Embedder`](kebab_core::Embedder).
|
||||
//!
|
||||
//! Runs an XLM-RoBERTa-large embedding model through `candle`
|
||||
//! (`candle-transformers`' XLM-RoBERTa) instead of onnxruntime. Two models
|
||||
//! are wired through a small **registry** ([`MODEL_REGISTRY`]):
|
||||
//!
|
||||
//! * `multilingual-e5-large` — the same weights the default
|
||||
//! [`FastembedEmbedder`](kebab_embed_local) uses (mean pooling,
|
||||
//! `query: `/`passage: ` prefixes). candle is the NUMA-safe drop-in:
|
||||
//! fastembed 4.9's onnxruntime hard-codes 48 intra-op threads, which
|
||||
//! corrupts the heap (double-free) on dual-socket NUMA hosts. candle's
|
||||
//! CPU backend sizes its threads off the global rayon pool, so a one-shot
|
||||
//! [`rayon::ThreadPoolBuilder`] cap (config `num_threads` / env
|
||||
//! `KEBAB_EMBED_THREADS`) keeps the worker count NUMA-safe.
|
||||
//! * `snowflake-arctic-embed-l-v2.0` — Snowflake's arctic-embed v2.0
|
||||
//! (CLS pooling, `query: ` on queries / no prefix on documents). Same
|
||||
//! XLM-RoBERTa-large architecture, dim 1024, so it rides the exact same
|
||||
//! tokenize → forward → L2 pipeline; only the pooling step and prefixes
|
||||
//! differ (both keyed off the per-model [`EmbedModelSpec`]).
|
||||
//!
|
||||
//! Output parity with the onnxruntime path (for e5) was proven by the
|
||||
//! Phase 0 spike (cosine 1.000000); the arctic path's pooling/prefix
|
||||
//! correctness is pinned by an `#[ignore]`d cosine>0.99 cross-check against
|
||||
//! Ollama's `snowflake-arctic-embed2` (see `tests/arctic_ollama_parity.rs`).
|
||||
//! The shared pipeline:
|
||||
//!
|
||||
//! 1. instruction prefix per [`EmbedModelSpec`] (query/doc);
|
||||
//! 2. tokenize (max_len 512, batch-longest padding, special tokens);
|
||||
//! 3. XLM-RoBERTa forward on the selected [`Device`];
|
||||
//! 4. pooling — mean (attention-mask-weighted) or CLS (first token);
|
||||
//! 5. L2 normalization.
|
||||
//!
|
||||
//! Model files (`config.json`, `tokenizer.json`, `model.safetensors`) are
|
||||
//! fetched via `hf-hub` into `{config.storage.model_dir}/candle/` (hf-hub's
|
||||
//! cache layout namespaces by repo, so e5 and arctic never collide).
|
||||
//!
|
||||
//! This crate is **opt-in** (`config.models.embedding.provider = "candle"`);
|
||||
//! the default provider stays `fastembed`. See
|
||||
//! `docs/superpowers/specs/2026-06-01-embed-candle-track-spec.md` and
|
||||
//! `docs/superpowers/specs/2026-06-03-arctic-embedder-spec.md`.
|
||||
|
||||
use std::sync::Mutex;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use candle_core::{DType, Device, Tensor};
|
||||
use candle_nn::VarBuilder;
|
||||
use candle_transformers::models::xlm_roberta::{Config as XlmConfig, XLMRobertaModel};
|
||||
use kebab_config::{Config, expand_path};
|
||||
use kebab_core::{Embedder, EmbeddingInput, EmbeddingKind, EmbeddingModelId, EmbeddingVersion};
|
||||
use tokenizers::{PaddingParams, PaddingStrategy, Tokenizer, TruncationParams};
|
||||
|
||||
/// Subdirectory under `config.storage.model_dir` where the candle adapter
|
||||
/// caches safetensors + tokenizer. Mirrors `kebab-embed-local`'s
|
||||
/// `fastembed/` subdir so the two backends never collide.
|
||||
const CANDLE_CACHE_SUBDIR: &str = "candle";
|
||||
|
||||
/// Token truncation length (both e5 and arctic-embed-l-v2.0 train at 512).
|
||||
const MAX_LEN: usize = 512;
|
||||
|
||||
/// Env var that overrides `config.models.embedding.num_threads`. Read once in
|
||||
/// [`CandleEmbedder::new`]; `0`/unset/unparseable means "leave rayon default".
|
||||
const ENV_EMBED_THREADS: &str = "KEBAB_EMBED_THREADS";
|
||||
|
||||
/// Pooling strategy over the model's last hidden state. Keyed per-model by
|
||||
/// [`EmbedModelSpec::pooling`] — e5 is mean, arctic is CLS.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Pooling {
|
||||
/// Attention-mask-weighted mean over all tokens (e5 / sentence-transformers
|
||||
/// `pooling_mode_mean_tokens`).
|
||||
Mean,
|
||||
/// First token (`<s>`/`[CLS]`) hidden state (arctic-embed v2.0 —
|
||||
/// `1_Pooling/config.json` has `pooling_mode_cls_token: true`).
|
||||
Cls,
|
||||
}
|
||||
|
||||
/// One supported embedding model: the HF repo candle downloads, the pooling
|
||||
/// strategy, and the e5-style instruction prefixes. [`MODEL_REGISTRY`] maps a
|
||||
/// `config.models.embedding.model` value to one of these.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct EmbedModelSpec {
|
||||
/// The short `config.models.embedding.model` value that selects this spec.
|
||||
pub name: &'static str,
|
||||
/// HuggingFace repo id candle fetches `config.json` / `tokenizer.json` /
|
||||
/// `model.safetensors` from.
|
||||
pub hf_repo: &'static str,
|
||||
/// Pooling over the last hidden state.
|
||||
pub pooling: Pooling,
|
||||
/// Prefix prepended to **query** inputs before tokenization.
|
||||
pub query_prefix: &'static str,
|
||||
/// Prefix prepended to **document** inputs before tokenization (arctic
|
||||
/// uses `""` — documents are embedded raw).
|
||||
pub doc_prefix: &'static str,
|
||||
/// Expected embedding dimension (model hidden size).
|
||||
pub dim: usize,
|
||||
/// Suffix folded into `model_version` so switching **to** this model
|
||||
/// triggers the `embedding_version` cascade even if the operator forgets
|
||||
/// to bump `config.version`. `None` keeps the bare `config.version` — used
|
||||
/// by e5 so candle-e5 and fastembed-e5 report the *same* version and stay
|
||||
/// interchangeable (the NUMA drop-in invariant — Phase 0 cosine 1.0).
|
||||
pub version_tag: Option<&'static str>,
|
||||
}
|
||||
|
||||
/// The models the candle adapter can load. Adding a model = one entry here
|
||||
/// (plus, for a non-XLM-R architecture, a new forward path — both current
|
||||
/// entries are XLM-RoBERTa-large so they share everything but pooling/prefix).
|
||||
static MODEL_REGISTRY: &[EmbedModelSpec] = &[
|
||||
EmbedModelSpec {
|
||||
name: "multilingual-e5-large",
|
||||
hf_repo: "intfloat/multilingual-e5-large",
|
||||
pooling: Pooling::Mean,
|
||||
query_prefix: "query: ",
|
||||
doc_prefix: "passage: ",
|
||||
dim: 1024,
|
||||
version_tag: None,
|
||||
},
|
||||
EmbedModelSpec {
|
||||
name: "snowflake-arctic-embed-l-v2.0",
|
||||
hf_repo: "Snowflake/snowflake-arctic-embed-l-v2.0",
|
||||
pooling: Pooling::Cls,
|
||||
query_prefix: "query: ",
|
||||
doc_prefix: "",
|
||||
dim: 1024,
|
||||
version_tag: Some("arctic-cls"),
|
||||
},
|
||||
];
|
||||
|
||||
/// Look up a model spec by `config.models.embedding.model`. Accepts either the
|
||||
/// short `name` or the full `hf_repo` id (mirrors the old e5 guard, which
|
||||
/// accepted both `multilingual-e5-large` and `intfloat/multilingual-e5-large`).
|
||||
pub(crate) fn lookup_spec(model: &str) -> Option<&'static EmbedModelSpec> {
|
||||
MODEL_REGISTRY
|
||||
.iter()
|
||||
.find(|s| s.name == model || s.hf_repo == model)
|
||||
}
|
||||
|
||||
/// Comma-separated list of supported model names, for the
|
||||
/// unsupported-model error message.
|
||||
fn supported_models() -> String {
|
||||
MODEL_REGISTRY
|
||||
.iter()
|
||||
.map(|s| s.name)
|
||||
.collect::<Vec<_>>()
|
||||
.join("`, `")
|
||||
}
|
||||
|
||||
/// Pure-Rust candle adapter. Construct via [`CandleEmbedder::new`]; the
|
||||
/// constructor downloads the model on first use, so share one instance.
|
||||
pub struct CandleEmbedder {
|
||||
// candle's `forward` is `&self`, but `XLMRobertaModel` is not guaranteed
|
||||
// `Sync`; the `Mutex` both supplies that bound and serializes inference
|
||||
// (callers batch sequentially anyway — same rationale as
|
||||
// `FastembedEmbedder`).
|
||||
model: Mutex<XLMRobertaModel>,
|
||||
tokenizer: Tokenizer,
|
||||
device: Device,
|
||||
/// The resolved model spec (pooling + prefixes) — drives `embed` and
|
||||
/// `embed_batch`.
|
||||
spec: &'static EmbedModelSpec,
|
||||
model_id: EmbeddingModelId,
|
||||
version: EmbeddingVersion,
|
||||
dimensions: usize,
|
||||
batch_size: usize,
|
||||
}
|
||||
|
||||
impl CandleEmbedder {
|
||||
/// Build an embedder from `Config`. Resolves the model spec from
|
||||
/// `config.models.embedding.model`, applies the NUMA thread cap, fetches
|
||||
/// the model into `{model_dir}/candle/`, and validates that the model's
|
||||
/// hidden size matches `config.models.embedding.dimensions` before
|
||||
/// returning.
|
||||
pub fn new(config: &Config) -> Result<Self> {
|
||||
// 1. NUMA thread cap. env `KEBAB_EMBED_THREADS` wins over the config
|
||||
// field; `0`/unset leaves rayon's default. `build_global` errors if
|
||||
// the pool was already initialized — intentionally ignored so a
|
||||
// second embedder (or a prior rayon user) is a no-op, not a failure.
|
||||
let n_threads = std::env::var(ENV_EMBED_THREADS)
|
||||
.ok()
|
||||
.and_then(|v| v.parse::<usize>().ok())
|
||||
.unwrap_or(config.models.embedding.num_threads as usize);
|
||||
if n_threads > 0 {
|
||||
if apply_thread_cap(n_threads) {
|
||||
tracing::info!(
|
||||
target: "kebab-embed-candle",
|
||||
num_threads = n_threads,
|
||||
"capped global rayon pool for candle CPU backend"
|
||||
);
|
||||
} else {
|
||||
tracing::debug!(
|
||||
target: "kebab-embed-candle",
|
||||
requested = n_threads,
|
||||
"global rayon pool already initialized; thread cap not applied"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// 1b. Model registry lookup. If the operator configured a model the
|
||||
// candle adapter doesn't know, fail fast (BEFORE the ~2GB
|
||||
// download) — never silently download one model and then label its
|
||||
// vectors with another name via `model_id()`, which would mislabel
|
||||
// `embedding_version` and corrupt a mixed index.
|
||||
let want = config.models.embedding.model.as_str();
|
||||
let spec = lookup_spec(want).ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"candle provider supports the models `{}`, but \
|
||||
config.models.embedding.model = '{want}'. Use provider=fastembed \
|
||||
for other models, or pick a supported one.",
|
||||
supported_models()
|
||||
)
|
||||
})?;
|
||||
|
||||
// 2. Resolve `{data_dir}/models/candle/` exactly like the fastembed
|
||||
// adapter resolves its own subdir.
|
||||
let data_dir = expand_path(&config.storage.data_dir, "");
|
||||
let model_dir = expand_path(&config.storage.model_dir, &data_dir.to_string_lossy());
|
||||
let cache_dir = model_dir.join(CANDLE_CACHE_SUBDIR);
|
||||
std::fs::create_dir_all(&cache_dir)
|
||||
.with_context(|| format!("create candle cache dir {}", cache_dir.display()))?;
|
||||
|
||||
let device = select_device();
|
||||
|
||||
// 3. Fetch model files via hf-hub into the candle cache.
|
||||
tracing::info!(
|
||||
target: "kebab-embed-candle",
|
||||
cache_dir = %cache_dir.display(),
|
||||
model = spec.hf_repo,
|
||||
pooling = ?spec.pooling,
|
||||
"loading candle embedding model (first run downloads ~2GB safetensors)"
|
||||
);
|
||||
let api = hf_hub::api::sync::ApiBuilder::new()
|
||||
.with_cache_dir(cache_dir.clone())
|
||||
.build()
|
||||
.context("kb-embed-candle: build hf-hub api")?;
|
||||
let repo = api.model(spec.hf_repo.to_string());
|
||||
let config_path = repo.get("config.json").context("download config.json")?;
|
||||
let tokenizer_path = repo
|
||||
.get("tokenizer.json")
|
||||
.context("download tokenizer.json")?;
|
||||
let weights_path = repo
|
||||
.get("model.safetensors")
|
||||
.context("download model.safetensors")?;
|
||||
|
||||
// 4. Build the candle XLM-RoBERTa model.
|
||||
let cfg_json = std::fs::read_to_string(&config_path)
|
||||
.with_context(|| format!("read {}", config_path.display()))?;
|
||||
let cfg: XlmConfig =
|
||||
serde_json::from_str(&cfg_json).context("kb-embed-candle: parse XLM-R config")?;
|
||||
|
||||
// Validate dim BEFORE building the model so a misconfigured
|
||||
// `dimensions` fails cheaply (matches FastembedEmbedder's contract).
|
||||
check_dim(cfg.hidden_size, config.models.embedding.dimensions)?;
|
||||
|
||||
let vb = unsafe {
|
||||
VarBuilder::from_mmaped_safetensors(&[weights_path], DType::F32, &device)
|
||||
.context("kb-embed-candle: mmap safetensors")?
|
||||
};
|
||||
let model =
|
||||
XLMRobertaModel::new(&cfg, vb).context("kb-embed-candle: build XLMRobertaModel")?;
|
||||
|
||||
let mut tokenizer = Tokenizer::from_file(&tokenizer_path)
|
||||
.map_err(|e| anyhow::anyhow!("kb-embed-candle: load tokenizer: {e}"))?;
|
||||
tokenizer
|
||||
.with_padding(Some(PaddingParams {
|
||||
strategy: PaddingStrategy::BatchLongest,
|
||||
..Default::default()
|
||||
}))
|
||||
.with_truncation(Some(TruncationParams {
|
||||
max_length: MAX_LEN,
|
||||
..Default::default()
|
||||
}))
|
||||
.map_err(|e| anyhow::anyhow!("kb-embed-candle: set truncation: {e}"))?;
|
||||
|
||||
// model_version: fold the model tag in for non-e5 models so a switch
|
||||
// triggers the embedding_version cascade; e5 keeps the bare
|
||||
// config.version to stay interchangeable with fastembed-e5.
|
||||
let version = match spec.version_tag {
|
||||
Some(tag) => {
|
||||
EmbeddingVersion(format!("{}+{}", config.models.embedding.version, tag))
|
||||
}
|
||||
None => EmbeddingVersion(config.models.embedding.version.clone()),
|
||||
};
|
||||
|
||||
tracing::info!(
|
||||
target: "kebab-embed-candle",
|
||||
dimensions = cfg.hidden_size,
|
||||
layers = cfg.num_hidden_layers,
|
||||
model = spec.name,
|
||||
"candle embedding model loaded"
|
||||
);
|
||||
|
||||
Ok(Self {
|
||||
model: Mutex::new(model),
|
||||
tokenizer,
|
||||
device,
|
||||
spec,
|
||||
model_id: EmbeddingModelId(config.models.embedding.model.clone()),
|
||||
version,
|
||||
dimensions: cfg.hidden_size,
|
||||
batch_size: config.models.embedding.batch_size.max(1),
|
||||
})
|
||||
}
|
||||
|
||||
/// Embed one batch of **already-prefixed** strings (the per-model prefix
|
||||
/// is applied by the caller [`CandleEmbedder::embed`]) through the candle
|
||||
/// pipeline: tokenize → forward → pool (mean|CLS) → L2.
|
||||
fn embed_batch(&self, prefixed: &[String]) -> Result<Vec<Vec<f32>>> {
|
||||
let encodings = self
|
||||
.tokenizer
|
||||
.encode_batch(prefixed.to_vec(), true)
|
||||
.map_err(|e| anyhow::anyhow!("kb-embed-candle: encode_batch: {e}"))?;
|
||||
|
||||
let bsz = encodings.len();
|
||||
// `embed` already returns early on empty input and `.chunks()` never
|
||||
// yields an empty slice, so this is currently unreachable — but guard
|
||||
// the index so a future refactor can't turn it into a panic.
|
||||
let Some(first) = encodings.first() else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
let seq = first.get_ids().len();
|
||||
|
||||
let mut ids = Vec::with_capacity(bsz * seq);
|
||||
let mut mask = Vec::with_capacity(bsz * seq);
|
||||
for enc in &encodings {
|
||||
ids.extend(enc.get_ids().iter().copied());
|
||||
mask.extend(enc.get_attention_mask().iter().map(|&m| m as f32));
|
||||
}
|
||||
|
||||
let input_ids = Tensor::from_vec(ids, (bsz, seq), &self.device)?;
|
||||
let attn_f32 = Tensor::from_vec(mask, (bsz, seq), &self.device)?;
|
||||
let token_type_ids = input_ids.zeros_like()?;
|
||||
|
||||
let hidden = {
|
||||
let guard = self
|
||||
.model
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
// forward: (input_ids, attention_mask, token_type_ids, past,
|
||||
// encoder_hidden, encoder_mask)
|
||||
guard.forward(&input_ids, &attn_f32, &token_type_ids, None, None, None)?
|
||||
};
|
||||
|
||||
// Pooling — per the model spec.
|
||||
let pooled = match self.spec.pooling {
|
||||
Pooling::Mean => {
|
||||
// attention-mask-weighted mean pooling
|
||||
let mask3 = attn_f32.unsqueeze(2)?; // (b, seq, 1)
|
||||
let summed = hidden.broadcast_mul(&mask3)?.sum(1)?; // (b, hidden)
|
||||
// counts ≥ 1 always: every input is prefixed AND special
|
||||
// tokens are added (encode_batch(_, true)), so no row has an
|
||||
// all-zero mask. If that invariant ever breaks, broadcast_div
|
||||
// would emit NaN vectors.
|
||||
let counts = mask3.sum(1)?; // (b, 1)
|
||||
summed.broadcast_div(&counts)?
|
||||
}
|
||||
Pooling::Cls => {
|
||||
// CLS pooling: the first token's hidden state. arctic-embed
|
||||
// v2.0 prepends `<s>` (the XLM-R BOS/CLS) at index 0, so
|
||||
// `hidden[:, 0, :]` is the sentence embedding.
|
||||
hidden.narrow(1, 0, 1)?.squeeze(1)? // (b, hidden)
|
||||
}
|
||||
};
|
||||
|
||||
// L2 normalize
|
||||
let norm = pooled.sqr()?.sum_keepdim(1)?.sqrt()?;
|
||||
let normalized = pooled.broadcast_div(&norm)?;
|
||||
|
||||
// `.contiguous()` before host copy: broadcast ops can leave a strided
|
||||
// view, which `to_vec2` rejects on the Metal backend (CPU tolerates it).
|
||||
Ok(normalized.contiguous()?.to_vec2::<f32>()?)
|
||||
}
|
||||
}
|
||||
|
||||
impl Embedder for CandleEmbedder {
|
||||
fn model_id(&self) -> EmbeddingModelId {
|
||||
self.model_id.clone()
|
||||
}
|
||||
|
||||
fn model_version(&self) -> EmbeddingVersion {
|
||||
self.version.clone()
|
||||
}
|
||||
|
||||
fn dimensions(&self) -> usize {
|
||||
self.dimensions
|
||||
}
|
||||
|
||||
fn embed(&self, inputs: &[EmbeddingInput<'_>]) -> Result<Vec<Vec<f32>>> {
|
||||
if inputs.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
|
||||
// Per-model instruction prefix BEFORE tokenization (same convention as
|
||||
// FastembedEmbedder for e5; arctic uses `query: `/no-prefix).
|
||||
let prefixed: Vec<String> = inputs.iter().map(|i| prefix_input(self.spec, i)).collect();
|
||||
|
||||
let mut out: Vec<Vec<f32>> = Vec::with_capacity(prefixed.len());
|
||||
for chunk in prefixed.chunks(self.batch_size) {
|
||||
let batch = self.embed_batch(chunk)?;
|
||||
for v in &batch {
|
||||
if v.len() != self.dimensions {
|
||||
anyhow::bail!(
|
||||
"candle returned vector of length {} but adapter expects {}",
|
||||
v.len(),
|
||||
self.dimensions
|
||||
);
|
||||
}
|
||||
}
|
||||
out.extend(batch);
|
||||
}
|
||||
|
||||
debug_assert_eq!(out.len(), inputs.len());
|
||||
Ok(out)
|
||||
}
|
||||
}
|
||||
|
||||
/// Build the prefixed string for one [`EmbeddingInput`] using the model spec.
|
||||
/// Free function so a unit test can pin the format without loading the model.
|
||||
/// For e5 this is byte-identical to `kebab-embed-local`'s `prefix_input` — the
|
||||
/// two backends MUST agree there or their vectors diverge.
|
||||
fn prefix_input(spec: &EmbedModelSpec, input: &EmbeddingInput<'_>) -> String {
|
||||
match input.kind {
|
||||
EmbeddingKind::Document => format!("{}{}", spec.doc_prefix, input.text),
|
||||
EmbeddingKind::Query => format!("{}{}", spec.query_prefix, input.text),
|
||||
}
|
||||
}
|
||||
|
||||
/// Select the compute device. Built with the `metal` feature (Apple Silicon
|
||||
/// GPU), try Metal and fall back to CPU on failure; otherwise CPU. Metal only
|
||||
/// compiles/runs on macOS — the Linux server builds the CPU path. Embedding
|
||||
/// vectors are model-defined, so Metal-produced and CPU-produced embeddings
|
||||
/// are cross-compatible (a Mac can ingest on GPU, the server query on CPU).
|
||||
fn select_device() -> Device {
|
||||
#[cfg(feature = "metal")]
|
||||
{
|
||||
match Device::new_metal(0) {
|
||||
Ok(d) => {
|
||||
tracing::info!(target: "kebab-embed-candle", "candle device = Metal (GPU)");
|
||||
return d;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
target: "kebab-embed-candle",
|
||||
error = %e,
|
||||
"Metal device unavailable; falling back to CPU"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
tracing::info!(target: "kebab-embed-candle", "candle device = CPU");
|
||||
Device::Cpu
|
||||
}
|
||||
|
||||
/// Apply a one-shot global rayon thread cap (the NUMA-safety lever). Returns
|
||||
/// `true` if this call set the pool, `false` if it was already initialized
|
||||
/// (cap not applied) or `n_threads == 0`. `#[doc(hidden)] pub` so the
|
||||
/// thread-cap test can drive it without loading the 2GB model.
|
||||
#[doc(hidden)]
|
||||
pub fn apply_thread_cap(n_threads: usize) -> bool {
|
||||
if n_threads == 0 {
|
||||
return false;
|
||||
}
|
||||
rayon::ThreadPoolBuilder::new()
|
||||
.num_threads(n_threads)
|
||||
.build_global()
|
||||
.is_ok()
|
||||
}
|
||||
|
||||
/// Compare model hidden size against the configured dim. Extracted so a unit
|
||||
/// test can exercise the error branch without loading the model.
|
||||
pub(crate) fn check_dim(model_dim: usize, cfg_dim: usize) -> Result<()> {
|
||||
if model_dim != cfg_dim {
|
||||
anyhow::bail!(
|
||||
"dimension mismatch: model={model_dim}, config={cfg_dim}; \
|
||||
update `config.models.embedding.dimensions` to match the model \
|
||||
(or pick a different model)."
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn e5_spec() -> &'static EmbedModelSpec {
|
||||
lookup_spec("multilingual-e5-large").expect("e5 in registry")
|
||||
}
|
||||
|
||||
fn arctic_spec() -> &'static EmbedModelSpec {
|
||||
lookup_spec("snowflake-arctic-embed-l-v2.0").expect("arctic in registry")
|
||||
}
|
||||
|
||||
// ── registry ─────────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn registry_resolves_e5_by_name_and_hf_repo() {
|
||||
assert_eq!(
|
||||
lookup_spec("multilingual-e5-large").map(|s| s.name),
|
||||
Some("multilingual-e5-large")
|
||||
);
|
||||
assert_eq!(
|
||||
lookup_spec("intfloat/multilingual-e5-large").map(|s| s.name),
|
||||
Some("multilingual-e5-large")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registry_resolves_arctic_and_its_pooling_is_cls() {
|
||||
let s = arctic_spec();
|
||||
assert_eq!(s.name, "snowflake-arctic-embed-l-v2.0");
|
||||
assert_eq!(s.hf_repo, "Snowflake/snowflake-arctic-embed-l-v2.0");
|
||||
assert_eq!(s.pooling, Pooling::Cls);
|
||||
assert_eq!(s.dim, 1024);
|
||||
assert_eq!(s.version_tag, Some("arctic-cls"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registry_e5_is_mean_pooling_no_version_tag() {
|
||||
let s = e5_spec();
|
||||
assert_eq!(s.pooling, Pooling::Mean);
|
||||
assert_eq!(s.version_tag, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registry_rejects_unknown_model() {
|
||||
assert!(lookup_spec("multilingual-e5-small").is_none());
|
||||
}
|
||||
|
||||
// ── prefix_input ─────────────────────────────────────────────────
|
||||
// e5 prefixes MUST match kebab-embed-local::prefix_input or candle vs
|
||||
// fastembed parity breaks; arctic uses query-only prefixing.
|
||||
|
||||
#[test]
|
||||
fn e5_prefix_document_uses_passage() {
|
||||
let input = EmbeddingInput {
|
||||
text: "hello world",
|
||||
kind: EmbeddingKind::Document,
|
||||
};
|
||||
assert_eq!(prefix_input(e5_spec(), &input), "passage: hello world");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn e5_prefix_query_uses_query() {
|
||||
let input = EmbeddingInput {
|
||||
text: "hello world",
|
||||
kind: EmbeddingKind::Query,
|
||||
};
|
||||
assert_eq!(prefix_input(e5_spec(), &input), "query: hello world");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn arctic_prefix_query_uses_query_doc_is_bare() {
|
||||
let doc = EmbeddingInput {
|
||||
text: "후입선출 자료구조",
|
||||
kind: EmbeddingKind::Document,
|
||||
};
|
||||
let qry = EmbeddingInput {
|
||||
text: "스택 자료구조",
|
||||
kind: EmbeddingKind::Query,
|
||||
};
|
||||
// arctic: documents are embedded raw, queries get `query: `.
|
||||
assert_eq!(prefix_input(arctic_spec(), &doc), "후입선출 자료구조");
|
||||
assert_eq!(prefix_input(arctic_spec(), &qry), "query: 스택 자료구조");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prefix_handles_empty_text() {
|
||||
let doc = EmbeddingInput {
|
||||
text: "",
|
||||
kind: EmbeddingKind::Document,
|
||||
};
|
||||
let qry = EmbeddingInput {
|
||||
text: "",
|
||||
kind: EmbeddingKind::Query,
|
||||
};
|
||||
assert_eq!(prefix_input(e5_spec(), &doc), "passage: ");
|
||||
assert_eq!(prefix_input(e5_spec(), &qry), "query: ");
|
||||
assert_eq!(prefix_input(arctic_spec(), &doc), "");
|
||||
assert_eq!(prefix_input(arctic_spec(), &qry), "query: ");
|
||||
}
|
||||
|
||||
// ── check_dim ────────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn check_dim_passes_for_1024() {
|
||||
check_dim(1024, 1024).expect("matching dims must pass");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn check_dim_rejects_384_vs_1024() {
|
||||
let err = check_dim(384, 1024).expect_err("dim mismatch must error");
|
||||
let msg = format!("{err}");
|
||||
assert!(
|
||||
msg.contains("384") && msg.contains("1024"),
|
||||
"error must mention both dims, got: {msg}"
|
||||
);
|
||||
}
|
||||
|
||||
// ── model guard ──────────────────────────────────────────────────
|
||||
// A model name not in the registry must fail fast (BEFORE the ~2GB
|
||||
// download), so we never download one model yet label its vectors with
|
||||
// another name via model_id() — which would mislabel embedding_version.
|
||||
|
||||
#[test]
|
||||
fn new_rejects_unsupported_model() {
|
||||
let mut config = kebab_config::Config::defaults();
|
||||
config.models.embedding.model = "multilingual-e5-small".to_string();
|
||||
// num_threads defaults to 0, so no global rayon side effect here.
|
||||
// `.err()` (not `expect_err`) avoids requiring `CandleEmbedder: Debug`
|
||||
// — it holds a Mutex/Tokenizer and intentionally derives no Debug.
|
||||
let err = CandleEmbedder::new(&config)
|
||||
.err()
|
||||
.expect("unsupported model must error");
|
||||
let msg = format!("{err:#}");
|
||||
assert!(
|
||||
msg.contains("candle provider supports the models"),
|
||||
"expected model-registry error, got: {msg}"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,128 +0,0 @@
|
||||
//! arctic-embed-l-v2.0 correctness gate (`#[ignore]` — needs the ~2GB candle
|
||||
//! model + a live Ollama serving `snowflake-arctic-embed2`).
|
||||
//!
|
||||
//! This is the load-bearing pooling/prefix check for the arctic integration.
|
||||
//! The recall measurement that justified adopting arctic (recall@10 130/132)
|
||||
//! went through Ollama's `snowflake-arctic-embed2`. The candle path
|
||||
//! re-implements the model (XLM-RoBERTa-large + **CLS** pooling + `query: ` on
|
||||
//! queries / no prefix on documents). If candle's pooling or prefix is wrong,
|
||||
//! its vectors silently diverge from the measured route and the 130 number
|
||||
//! does NOT carry over. This test pins them together: per-sentence cosine
|
||||
//! between the candle vector and the Ollama vector must be **> 0.99**.
|
||||
//!
|
||||
//! `#[ignore]` because it depends on an external Ollama daemon (CI is
|
||||
//! headless/offline). The leader MUST run it once before merge.
|
||||
//!
|
||||
//! ## Manual run
|
||||
//!
|
||||
//! 1. Confirm Ollama is reachable and has the model:
|
||||
//! ```sh
|
||||
//! curl -s http://192.168.0.47:11434/api/tags # should list snowflake-arctic-embed2
|
||||
//! ```
|
||||
//! 2. Run (downloads the ~2GB candle safetensors on first run):
|
||||
//! ```sh
|
||||
//! CARGO_TARGET_DIR=/build/out/cargo-target \
|
||||
//! KEBAB_ARCTIC_OLLAMA_ENDPOINT=http://192.168.0.47:11434 \
|
||||
//! cargo test -p kebab-embed-candle --test arctic_ollama_parity -- --ignored --nocapture
|
||||
//! ```
|
||||
//! The endpoint defaults to `http://192.168.0.47:11434` if the env is unset.
|
||||
//!
|
||||
//! Record the printed `ARCTIC_PARITY_SUMMARY cosine_min=...` in
|
||||
//! `/tmp/arctic-result.md` + `tasks/HOTFIXES.md`.
|
||||
|
||||
use kebab_config::Config;
|
||||
use kebab_core::{Embedder, EmbeddingInput, EmbeddingKind};
|
||||
use kebab_embed_candle::CandleEmbedder;
|
||||
use kebab_embed_ollama::OllamaEmbedder;
|
||||
|
||||
const DOGFOOD_CONFIG: &str = "/build/dogfood/config.toml";
|
||||
const DEFAULT_OLLAMA_ENDPOINT: &str = "http://192.168.0.47:11434";
|
||||
|
||||
/// Mixed Korean / English + the descriptive-recall shapes arctic was adopted
|
||||
/// for (synonym / abbreviation / English term). Covers both prefix paths.
|
||||
const SENTENCES: &[&str] = &[
|
||||
"스택 자료구조",
|
||||
"후입선출 방식으로 동작하는 자료구조",
|
||||
"큐는 선입선출 자료구조이다",
|
||||
"Rust ownership and the borrow checker",
|
||||
"소유권과 빌림 검사기는 메모리 안전성을 보장한다",
|
||||
"SVM 은 support vector machine 의 약자이다",
|
||||
"정렬 알고리즘의 시간 복잡도",
|
||||
"The capital of France is Paris.",
|
||||
];
|
||||
|
||||
fn cosine(a: &[f32], b: &[f32]) -> f32 {
|
||||
let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
|
||||
let na: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
let nb: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
dot / (na * nb)
|
||||
}
|
||||
|
||||
/// Base config: prefer the canonical dogfood config (for storage/cache roots),
|
||||
/// fall back to `Config::defaults()` so the test still runs on a bare clone.
|
||||
fn base_config() -> Config {
|
||||
Config::load(Some(std::path::Path::new(DOGFOOD_CONFIG))).unwrap_or_else(|_| Config::defaults())
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "needs ~2GB candle model + live Ollama (snowflake-arctic-embed2); run manually before merge"]
|
||||
fn candle_arctic_matches_ollama_arctic() {
|
||||
let endpoint = std::env::var("KEBAB_ARCTIC_OLLAMA_ENDPOINT")
|
||||
.unwrap_or_else(|_| DEFAULT_OLLAMA_ENDPOINT.to_string());
|
||||
|
||||
// candle side: the in-process arctic model.
|
||||
let mut candle_cfg = base_config();
|
||||
candle_cfg.models.embedding.provider = "candle".to_string();
|
||||
candle_cfg.models.embedding.model = "snowflake-arctic-embed-l-v2.0".to_string();
|
||||
candle_cfg.models.embedding.dimensions = 1024;
|
||||
|
||||
// Ollama side: the reference route the recall numbers came from.
|
||||
let mut ollama_cfg = base_config();
|
||||
ollama_cfg.models.embedding.provider = "ollama".to_string();
|
||||
ollama_cfg.models.embedding.model = "snowflake-arctic-embed2".to_string();
|
||||
ollama_cfg.models.embedding.dimensions = 1024;
|
||||
ollama_cfg.models.embedding.endpoint = Some(endpoint.clone());
|
||||
|
||||
let candle = CandleEmbedder::new(&candle_cfg).expect("build candle arctic embedder");
|
||||
let ollama = OllamaEmbedder::new(&ollama_cfg).expect("build ollama arctic embedder");
|
||||
|
||||
// Exercise BOTH prefix paths so a query-side divergence can't hide.
|
||||
let inputs: Vec<EmbeddingInput> = SENTENCES
|
||||
.iter()
|
||||
.flat_map(|s| {
|
||||
[EmbeddingKind::Document, EmbeddingKind::Query]
|
||||
.into_iter()
|
||||
.map(move |kind| EmbeddingInput { text: s, kind })
|
||||
})
|
||||
.collect();
|
||||
|
||||
let cv = candle.embed(&inputs).expect("candle embed");
|
||||
let ov = ollama
|
||||
.embed(&inputs)
|
||||
.expect("ollama embed (is snowflake-arctic-embed2 pulled @ the endpoint?)");
|
||||
|
||||
assert_eq!(cv.len(), ov.len(), "embedding counts must match");
|
||||
assert_eq!(cv.len(), inputs.len(), "one vector per input");
|
||||
assert_eq!(candle.dimensions(), 1024);
|
||||
|
||||
let mut min_cos = f32::INFINITY;
|
||||
for (i, inp) in inputs.iter().enumerate() {
|
||||
assert_eq!(cv[i].len(), 1024, "candle dim");
|
||||
assert_eq!(ov[i].len(), 1024, "ollama dim");
|
||||
let c = cosine(&cv[i], &ov[i]);
|
||||
min_cos = min_cos.min(c);
|
||||
let kind = match inp.kind {
|
||||
EmbeddingKind::Document => "doc",
|
||||
EmbeddingKind::Query => "qry",
|
||||
};
|
||||
let preview: String = inp.text.chars().take(36).collect();
|
||||
println!("[{i:>2}] {kind} cos={c:.6} {preview}");
|
||||
}
|
||||
|
||||
println!("ARCTIC_PARITY_SUMMARY cosine_min={min_cos:.6} endpoint={endpoint}");
|
||||
assert!(
|
||||
min_cos > 0.99,
|
||||
"candle arctic vs Ollama arctic cosine_min={min_cos:.6} ≤ 0.99 — \
|
||||
pooling/prefix mismatch; the recall=130 measurement will NOT reproduce"
|
||||
);
|
||||
}
|
||||
@@ -1,96 +0,0 @@
|
||||
//! Parity test (spec §7, `#[ignore]` — needs the ~2GB model + network).
|
||||
//!
|
||||
//! Confirms the candle backend reproduces the onnxruntime `FastembedEmbedder`
|
||||
//! vectors closely enough that no re-index is required (spec D-reindex):
|
||||
//! per-sentence cosine ≥ 0.9999, and reports the dimension-wise max absolute
|
||||
//! difference (the number the re-index decision hangs on).
|
||||
//!
|
||||
//! Run manually:
|
||||
//! CARGO_TARGET_DIR=/build/out/cargo-target/target \
|
||||
//! cargo test -p kebab-embed-candle --release -- --ignored --nocapture
|
||||
//!
|
||||
//! Uses the canonical dogfood config so both backends resolve the same model
|
||||
//! identifiers and cache roots.
|
||||
|
||||
use kebab_config::Config;
|
||||
use kebab_core::{Embedder, EmbeddingInput, EmbeddingKind};
|
||||
use kebab_embed_candle::CandleEmbedder;
|
||||
use kebab_embed_local::FastembedEmbedder;
|
||||
|
||||
const DOGFOOD_CONFIG: &str = "/build/dogfood/config.toml";
|
||||
|
||||
/// Mixed Korean / English parity set (≥ 8 sentences, mirrors the Phase 0 spike).
|
||||
const SENTENCES: &[&str] = &[
|
||||
"The quick brown fox jumps over the lazy dog.",
|
||||
"오늘 날씨가 정말 좋아서 산책을 나가고 싶다.",
|
||||
"Rust is a systems programming language focused on safety and performance.",
|
||||
"벡터 검색은 임베딩 사이의 코사인 유사도를 이용한다.",
|
||||
"Machine learning models require large amounts of training data.",
|
||||
"한국어와 영어가 섞인 문장도 멀티링구얼 모델은 잘 처리한다.",
|
||||
"The capital of France is Paris, a city known for its art and culture.",
|
||||
"이 프로젝트는 로컬 우선 지식 베이스와 검색 증강 생성을 목표로 한다.",
|
||||
"Database indexing dramatically speeds up query performance.",
|
||||
"임베딩 모델을 candle 로 옮기면 NUMA 서버에서 안전하게 돌릴 수 있다.",
|
||||
];
|
||||
|
||||
fn cosine(a: &[f32], b: &[f32]) -> f32 {
|
||||
let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
|
||||
let na: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
let nb: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
dot / (na * nb)
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "needs ~2GB model + network; run manually for the re-index decision"]
|
||||
fn candle_matches_fastembed() {
|
||||
let config = Config::load(Some(std::path::Path::new(DOGFOOD_CONFIG)))
|
||||
.expect("load dogfood config for parity baseline");
|
||||
|
||||
let candle = CandleEmbedder::new(&config).expect("build CandleEmbedder");
|
||||
let fastembed = FastembedEmbedder::new(&config).expect("build FastembedEmbedder");
|
||||
|
||||
// Cover BOTH prefix paths (`passage:` for Document, `query:` for Query) so
|
||||
// a query-side prefix/pooling divergence can't slip through (reviewer note).
|
||||
let inputs: Vec<EmbeddingInput> = SENTENCES
|
||||
.iter()
|
||||
.flat_map(|s| {
|
||||
[EmbeddingKind::Document, EmbeddingKind::Query]
|
||||
.into_iter()
|
||||
.map(move |kind| EmbeddingInput { text: s, kind })
|
||||
})
|
||||
.collect();
|
||||
|
||||
let cv = candle.embed(&inputs).expect("candle embed");
|
||||
let fv = fastembed.embed(&inputs).expect("fastembed embed");
|
||||
|
||||
assert_eq!(cv.len(), fv.len(), "embedding counts must match");
|
||||
assert_eq!(cv.len(), inputs.len(), "one vector per input");
|
||||
assert_eq!(candle.dimensions(), 1024);
|
||||
|
||||
let mut min_cos = f32::INFINITY;
|
||||
let mut max_abs_diff = 0f32;
|
||||
for (i, inp) in inputs.iter().enumerate() {
|
||||
assert_eq!(cv[i].len(), 1024, "candle dim");
|
||||
assert_eq!(fv[i].len(), 1024, "fastembed dim");
|
||||
let c = cosine(&cv[i], &fv[i]);
|
||||
min_cos = min_cos.min(c);
|
||||
let diff = cv[i]
|
||||
.iter()
|
||||
.zip(&fv[i])
|
||||
.map(|(a, b)| (a - b).abs())
|
||||
.fold(0f32, f32::max);
|
||||
max_abs_diff = max_abs_diff.max(diff);
|
||||
let kind = match inp.kind {
|
||||
EmbeddingKind::Document => "doc",
|
||||
EmbeddingKind::Query => "qry",
|
||||
};
|
||||
let preview: String = inp.text.chars().take(36).collect();
|
||||
println!("[{i:>2}] {kind} cos={c:.6} max_abs_diff={diff:.6e} {preview}");
|
||||
}
|
||||
|
||||
println!("PARITY_SUMMARY cosine_min={min_cos:.6} max_abs_diff={max_abs_diff:.6e}");
|
||||
assert!(
|
||||
min_cos >= 0.9999,
|
||||
"candle vs fastembed cosine_min={min_cos:.6} < 0.9999 — investigate before merge"
|
||||
);
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
//! Thread-cap test (spec §7). Own integration binary → clean process, so the
|
||||
//! one-shot global rayon pool is initialized exactly once, by us.
|
||||
//!
|
||||
//! Verifies that `apply_thread_cap(4)` sizes the global rayon pool to 4, which
|
||||
//! is the lever that keeps candle's CPU backend NUMA-safe (vs onnxruntime's
|
||||
//! hard-coded 48 intra-op threads).
|
||||
|
||||
use kebab_embed_candle::apply_thread_cap;
|
||||
|
||||
#[test]
|
||||
fn thread_cap_sizes_global_rayon_pool() {
|
||||
// Must run before any other rayon use in this process. As the only test in
|
||||
// this binary that touches rayon, that holds.
|
||||
let applied = apply_thread_cap(4);
|
||||
assert!(applied, "first build_global call should succeed");
|
||||
assert_eq!(
|
||||
rayon::current_num_threads(),
|
||||
4,
|
||||
"global rayon pool must be capped at the requested 4 threads"
|
||||
);
|
||||
|
||||
// A second cap attempt is a no-op (pool already built), not a panic.
|
||||
assert!(
|
||||
!apply_thread_cap(8),
|
||||
"second build_global must report not-applied"
|
||||
);
|
||||
assert_eq!(
|
||||
rayon::current_num_threads(),
|
||||
4,
|
||||
"thread count must stay at the first cap"
|
||||
);
|
||||
}
|
||||
@@ -4,11 +4,10 @@
|
||||
//!
|
||||
//! ## Why this exists
|
||||
//!
|
||||
//! The candle backend ([`kebab-embed-candle`]) runs arctic-embed-l-v2.0
|
||||
//! in-process (pure Rust, NUMA-safe). This crate is the **fallback** path:
|
||||
//! it offloads embedding to a local/remote Ollama daemon (`snowflake-arctic-embed2`),
|
||||
//! which is exactly the route the recall measurements used — so it reproduces
|
||||
//! the measured numbers (recall@10 130/132) byte-for-route. Opt-in via
|
||||
//! This crate offloads embedding to a local/remote Ollama daemon
|
||||
//! (`snowflake-arctic-embed2`), which is exactly the route the recall
|
||||
//! measurements used — so it reproduces the measured numbers (recall@10
|
||||
//! 130/132) byte-for-route. Opt-in via
|
||||
//! `config.models.embedding.provider = "ollama"`.
|
||||
//!
|
||||
//! ## Wire shape
|
||||
@@ -64,10 +63,9 @@ const REQUEST_TIMEOUT_SECS: u64 = 300;
|
||||
|
||||
/// Resolve the (query_prefix, doc_prefix) for an Ollama embedding model tag.
|
||||
///
|
||||
/// Mirrors `kebab-embed-candle`'s `MODEL_REGISTRY`, but keyed on the **Ollama
|
||||
/// model tag** (which differs from the HF id — e.g. `snowflake-arctic-embed2`
|
||||
/// vs `Snowflake/snowflake-arctic-embed-l-v2.0`). Kept here rather than shared
|
||||
/// so this crate does not depend on the candle backend.
|
||||
/// Resolve the (query_prefix, doc_prefix) for an Ollama embedding model tag,
|
||||
/// keyed on the **Ollama model tag** (which differs from the HF id — e.g.
|
||||
/// `snowflake-arctic-embed2` vs `Snowflake/snowflake-arctic-embed-l-v2.0`).
|
||||
///
|
||||
/// An unrecognized model gets no prefix (`("", "")`): many embedding models
|
||||
/// are not instruction-tuned, so embedding the raw text is the correct default
|
||||
|
||||
@@ -16,7 +16,7 @@ Cargo workspace, 함수 호출 기반 모듈러 모놀리스. UI binary (`kebab-
|
||||
| metadata | SQLite + FTS5 (lexical search + v0.20.1 한국어 형태소 tokenizer via lindera-ko-dic) |
|
||||
| vector | LanceDB (embedded, model 별 분리 table) |
|
||||
| Markdown parser | `pulldown-cmark`. frontmatter 에 title 없으면 첫 H1 → H2 → 첫 paragraph 80 자 → 파일명 순으로 자동 채움 (`parser_version = md-frontmatter-v2`, 기존 doc 도 다음 ingest 에서 갱신) |
|
||||
| embedding | `fastembed-rs` (`multilingual-e5-large`, 1024d, v0.18.0부터 default 업그레이드). opt-in 대안: candle (e5 또는 `snowflake-arctic-embed-l-v2.0`) / Ollama `/api/embed`. arctic = 설명형 query recall 보강 (v0.26.0, 아래 결정표) |
|
||||
| embedding | `fastembed-rs` (`multilingual-e5-large`, 1024d, v0.18.0부터 default 업그레이드). opt-in 대안: Ollama `/api/embed` (`snowflake-arctic-embed-l-v2.0` 등). arctic = 설명형 query recall 보강 (v0.26.0, 아래 결정표) |
|
||||
| 한국어 형태소분석 | `lindera-ko-dic` (FTS5 외부 tokenizer, v0.20.1) — 2자 이상 한국어 query 지원 |
|
||||
| LLM | Ollama HTTP (default `gemma4:e4b` ─ OCR / caption 와 family 통일. 사용자가 더 큰 variant `gemma4:26b` 등으로 override 가능) |
|
||||
| 음성 ASR | `whisper.cpp` (via `whisper-rs`) — P8 보류, 시스템 dep brainstorm 후 |
|
||||
@@ -68,7 +68,6 @@ flowchart TB
|
||||
subgraph Adapters ["traits + adapters"]
|
||||
embed["kebab-embed<br/>(trait)"]
|
||||
embedlocal["kebab-embed-local<br/>(fastembed, default)"]
|
||||
embedcandle["kebab-embed-candle<br/>(candle, e5+arctic, NUMA-safe opt-in)"]
|
||||
embedollama["kebab-embed-ollama<br/>(Ollama /api/embed, opt-in)"]
|
||||
llm["kebab-llm<br/>(trait)"]
|
||||
llmlocal["kebab-llm-local<br/>(Ollama)"]
|
||||
@@ -95,7 +94,6 @@ flowchart TB
|
||||
app --> sqlite
|
||||
app --> vector
|
||||
app --> embedlocal
|
||||
app --> embedcandle
|
||||
app --> embedollama
|
||||
app --> llmlocal
|
||||
app --> search
|
||||
@@ -109,8 +107,6 @@ flowchart TB
|
||||
paud --> core
|
||||
pcode --> core
|
||||
embedlocal --> embed
|
||||
embedcandle --> core
|
||||
embedcandle --> config
|
||||
embedollama --> core
|
||||
embedollama --> config
|
||||
llmlocal --> llm
|
||||
@@ -145,16 +141,13 @@ UI → store/llm/parse 직접 의존 금지. 모든 user-facing 진입은 `kebab
|
||||
|
||||
| provider | 모델 | pooling / prefix | 위치 | 언제 |
|
||||
|---|---|---|---|---|
|
||||
| `fastembed` (기본) | `multilingual-e5-large` | mean / `query:`·`passage:` | in-process (onnxruntime) | 기본. 단일 소켓 호스트 |
|
||||
| `candle` | e5 또는 `snowflake-arctic-embed-l-v2.0` | 모델별 (e5=mean, arctic=CLS) / arctic=`query:`·무접두어 | in-process (pure Rust) | NUMA 서버 (onnxruntime 48-스레드 double-free 회피), Apple Silicon Metal GPU |
|
||||
| `ollama` | `snowflake-arctic-embed2` 등 | 모델 태그로 추론 / arctic=`query:`·무접두어 | 원격 HTTP (`/api/embed`) | candle 폴백, 측정에 쓴 경로 그대로 재현 |
|
||||
| `fastembed` (기본) | `multilingual-e5-large` | mean / `query:`·`passage:` | in-process (onnxruntime) | 기본. 모든 호스트 |
|
||||
| `ollama` | `snowflake-arctic-embed2` 등 | 모델 태그로 추론 / arctic=`query:`·무접두어 | 원격 HTTP (`/api/embed`) | GPU 서버 위임, 측정에 쓴 경로 그대로 재현 |
|
||||
|
||||
**arctic-embed-l-v2.0 채택 근거**: 별칭(doc-side expansion) 제거(v0.25.0) 후 설명형
|
||||
query 의 recall 보강책. 측정(`/build/dogfood/logs/2026-06-03-method-measurements.md`)에서
|
||||
arctic = recall@10 130/132 (e5 대비 +7, 색인 1회·per-query 0·LLM 0, 용어 무손실).
|
||||
candle 이 주 백엔드(in-process, NUMA 안전), Ollama 가 폴백(측정 경로 재현). 두 경로의
|
||||
pooling/prefix 정확성은 `kebab-embed-candle/tests/arctic_ollama_parity.rs`
|
||||
(candle arctic vs Ollama arctic 코사인>0.99, `#[ignore]`) 로 고정. e5 → arctic 전환은
|
||||
Ollama 백엔드(`provider = "ollama"`)로 arctic 모델 사용. e5 → arctic 전환은
|
||||
`embedding_version` cascade (모델별 벡터 상이) → 재색인 필요. 기본값 e5 유지라 기존
|
||||
사용자 무영향. 자세한 내용: [tasks/HOTFIXES.md](../tasks/HOTFIXES.md) 2026-06-03 arctic entry.
|
||||
|
||||
@@ -206,8 +199,7 @@ kebab/
|
||||
│ ├── kebab-store-sqlite/ # SQLite + FTS5 (V001/V002/V003) (P1-6, P2-1, P3-3). src/derivation_cache.rs = derivation_cache 테이블 저장소 (V012, v0.21.0)
|
||||
│ ├── kebab-search/ # Lexical + Vector + Hybrid retriever (P2-2, P3-4)
|
||||
│ ├── kebab-embed/ kebab-embed-local/ # Embedder trait + fastembed adapter (P3-1, P3-2)
|
||||
│ ├── kebab-embed-candle/ # candle (pure-Rust) Embedder, 모델 레지스트리(e5 mean + arctic CLS), NUMA-safe opt-in provider=candle (Track 1, v0.22.0; arctic v0.26.0)
|
||||
│ ├── kebab-embed-ollama/ # Ollama /api/embed Embedder, opt-in provider=ollama (arctic 폴백 경로, v0.26.0)
|
||||
│ ├── kebab-embed-ollama/ # Ollama /api/embed Embedder, opt-in provider=ollama (arctic 경로, v0.26.0)
|
||||
│ ├── kebab-store-vector/ # LanceDB VectorStore (P3-3, P7-3 follow-up)
|
||||
│ ├── kebab-llm/ kebab-llm-local/ # LanguageModel trait + Ollama adapter (P4-1, P4-2)
|
||||
│ ├── kebab-rag/ # RAG pipeline (P4-3)
|
||||
|
||||
Reference in New Issue
Block a user