calternal-embed

Local ONNX embeddings and rebuildable semantic search index for calternal

Local semantic embeddings in rebuildable per-User SQLite files.

Model files live below the server-owned .system tree. Search uses the installed model only; a failed download or inference leaves Tantivy search available and does not stop server startup.

Source: crates/calternal-embed/src/lib.rs

Structs

PhotoClipHit

pub struct PhotoClipHit

One permission-checked image result before Photos adds its route.

Fields

• pub path: String: Data-directory-relative source path.
• pub owner_id: String: File owner. A shared photo may belong to another Home.
• pub item_id: String: Stable Files item identity.
• pub score: f32: Cosine similarity in the range -1 through 1.

Implements: Clone, Debug, PartialEq

Source: crates/calternal-embed/src/clip_store.rs:95

PhotoClipIndexer

pub struct PhotoClipIndexer

CLIP image index with bounded search and model access.

Implements: Clone

PhotoClipIndexer::start

pub async fn start(root: Root, database_path: impl AsRef<Path>) -> Result<Self, EmbedError>

Open the old migration source and start on-demand model verification. database_path comes from a held server-owned .system handle. Each User’s private file opens lazily below their Index directory (#435).

PhotoClipIndexer::start_with_backend

pub async fn start_with_backend(
        root: Root,
        database_path: impl AsRef<Path>,
        backend: Box<dyn ClipInferenceBackend>,
    ) -> Result<Self, EmbedError>

Open the same private vector stores with a server-selected inference backend. This keeps backend-neutral callers and deterministic contract tests independent of model download timing (§36, #895). Authority and vector validation are identical to start; no model worker is started. Keep the server-owned .system directory handle for database_path alive while the indexer is used, as with start.

PhotoClipIndexer::index_thumbnail

pub async fn index_thumbnail(
        &self,
        owner_id: &str,
        item_id: &str,
        path: &str,
        content_hash: &str,
        thumbnail: Vec<u8>,
    ) -> Result<(), EmbedError>

Index a Files-owned WebP thumbnail. The image and embedding are size-bounded; first use loads one shared model, and repeated content at the current revision is a no-op. The validated owner selects one file (#435, #503).

PhotoClipIndexer::needs_index

pub async fn needs_index(
        &self,
        maintenance: &PhotoIndexMaintenance,
        item_id: &str,
        content_hash: &str,
    ) -> Result<bool, EmbedError>

Check whether a photo needs a thumbnail read and inference. A path-only rename updates the derived route identity without embedding the same bytes again. The audited Files Index path selects one private file; this read API does not accept an owner ID (#458).

PhotoClipIndexer::remove_photo

pub async fn remove_photo(&self, owner_id: &str, item_id: &str) -> Result<(), EmbedError>

Remove one Derived vector after its authoritative Files Item disappears. The owner selects one private file (#435).

PhotoClipIndexer::search

pub async fn search(
        &self,
        authority: &crate::VectorReadCapability,
        query: &str,
        limit: usize,
    ) -> Result<Vec<PhotoClipHit>, EmbedError>

Search the viewer’s private file and exact Share subtrees only. A busy inference slot returns no CLIP results without queueing (#455).

Source: crates/calternal-embed/src/clip_store.rs:140

PhotoIndexMaintenance

pub struct PhotoIndexMaintenance

Audited maintenance read for a photo path selected by the Files Index job. The owner is parsed from the validated path; no read accepts an owner ID from a request or a caller chosen pool (#458).

PhotoIndexMaintenance::from_job_path

pub fn from_job_path(path: &RelPath) -> Result<Self, EmbedError>

Construct only from a data-directory-relative Files Index path. The job must have read that path from the authoritative Files Index (#458).

Source: crates/calternal-embed/src/clip_store.rs:109

SemanticHit

pub struct SemanticHit

One permission-checked vector result before the search crate adds a route.

Fields

• pub path: String: Data-directory-relative source path.
• pub title: String: Note title or file name.
• pub snippet: String: Plain-text preview from the best matching chunk.
• pub score: f32: Cosine similarity, where a higher value is a closer match.
• pub mime: Option<String>: Files Index type for the same stable path, used to build its client route.

Implements: Clone, Debug, PartialEq

Source: crates/calternal-embed/src/store.rs:57

SemanticIndexer

pub struct SemanticIndexer

Shared query handle and bounded update queue for semantic Search (#435, #1011).

Implements: Clone

SemanticIndexer::start

pub async fn start(root: Root, database_path: impl AsRef<Path>) -> Result<Self, EmbedError>

Open the old migration source and start private User vector workers.

database_path must be derived from the server’s held .system directory handle. Each User file opens lazily below their private Index directory. Model download does not delay HTTP startup (#435).

SemanticIndexer::start_deferred

pub async fn start_deferred(
        root: Root,
        database_path: impl AsRef<Path>,
    ) -> Result<(Self, SemanticStartup), EmbedError>

Create a lazy, bounded legacy pool before HTTP starts (#1011). The caller must run the returned startup phase after bind on a low-priority runtime with a bounded blocking pool. Schema cleanup, private migrations and inference stay off the listener’s path. SQLx creates its SQLite thread on first acquire, so it inherits that runtime thread’s lower CPU and I/O priority instead of the HTTP thread’s.

SemanticIndexer::notify_changed

pub fn notify_changed(&self, path: RelPath) -> Result<(), EmbedError>

Queue a file or Home after a server write. A full queue is reported so the caller can log it; the periodic reconcile will recover the update. The worker routes the path to one User’s private file (#435).

SemanticIndexer::notify_removed

pub fn notify_removed(&self, path: RelPath) -> Result<(), EmbedError>

Queue a deleted file or folder for vector removal. A whole-Home removal cannot reopen a deleted User’s private file (#435).

SemanticIndexer::purge_deleted_user

pub async fn purge_deleted_user(&self, user_id: &str) -> Result<(), EmbedError>

Finish account deletion without reopening the User’s purged Index. This also removes old shared rows when that User never triggered lazy migration. The caller runs this after the filesystem purge (#435).

SemanticIndexer::reconcile

pub async fn reconcile(&self) -> Result<(), EmbedError>

Reconcile every Home within its own private file and remove vectors for files that no longer exist. The first pass starts the model on demand (#435, #503).

SemanticIndexer::search

pub async fn search(
        &self,
        authority: &VectorReadCapability,
        query: &str,
        limit: usize,
    ) -> Result<Vec<SemanticHit>, EmbedError>

Search the viewer’s private file and exact subtrees authorized by Share capabilities. Short queries skip low-signal inference (#455).

Source: crates/calternal-embed/src/store.rs:72

SemanticStartup

pub struct SemanticStartup

Own the deferred schema upgrade and worker receivers (#1011). No vectors or model are touched until run executes on the server’s low-priority runtime. Dropping this value closes the work queues.

SemanticStartup::run

pub async fn run(self) -> Result<(), EmbedError>

Prepare the derived schema, then start one indexing and one model task. Queries return keyword-only results until cleanup succeeds; stale vectors are never exposed during a revision change (#1011, #122).

Source: crates/calternal-embed/src/store.rs:89

VectorReadCapability

pub struct VectorReadCapability

Read authority derived from the server’s authenticated Search context. Home and Share roots are separate capabilities: a Share query can select only its indexed subtree before reading any vectors (#458, #455, §48).

VectorReadCapability::from_search_context

pub fn from_search_context(context: &calternal_plugin::SearchContext) -> Option<Self>

Derive Home and Share authority from authenticated Search roots. A caller cannot select a vector file by supplying an owner ID (#458).

VectorReadCapability::from_request_context

pub fn from_request_context(context: &calternal_plugin::PluginRequestContext) -> Option<Self>

Plugin routes receive this context only from the authenticated server adapter; its User identity selects the private vector file (#458).

Source: crates/calternal-embed/src/lib.rs:22

Enums

EmbedError

pub enum EmbedError

Errors from the derived semantic index. The search provider logs these and keeps keyword search available when an embedding operation fails.

Variants

• Filesystem(#[from] calternal_fs::Error)
• Database(#[from] sqlx::Error)
• Model(String)
• WorkerStopped
• QueueFull
• BlockingTask(String)

Implements: Debug, Error

Source: crates/calternal-embed/src/lib.rs:150

Traits

ClipInferenceBackend

pub trait ClipInferenceBackend: Send

Inference operations required by the derived photo index.

Photos depends on this trait and vectors only. ONNX Runtime stays behind calternal-embed so another local backend can replace it without changing photo search or its durable index.

ClipInferenceBackend::embed_text

fn embed_text(&mut self, text: &str) -> Result<Vec<f32>, EmbedError>;

Return a unit length embedding for one text query.

ClipInferenceBackend::embed_image

fn embed_image(&mut self, pixels: &[f32]) -> Result<Vec<f32>, EmbedError>;

Return a unit length embedding for one RGB image tensor in NCHW layout.

ClipInferenceBackend::dimensions

fn dimensions(&self) -> usize;

Return the number of values in each embedding.

Source: crates/calternal-embed/src/clip_model.rs:25

Functions

is_searchable_text

pub fn is_searchable_text(mime: &str) -> bool

Shared MIME policy for keyword and semantic indexing (#851).

Source: crates/calternal-embed/src/lib.rs:141

load_photo_clip_model

pub async fn load_photo_clip_model(
    root: calternal_fs::Root,
) -> Result<Box<dyn ClipInferenceBackend>, EmbedError>

Load the checked-in CLIP model using the shared model download and verify path, returning only the backend-neutral inference interface.

Source: crates/calternal-embed/src/lib.rs:134

purge_deleted_user_clip

pub async fn purge_deleted_user_clip(root: &Root, owner_id: &str) -> Result<(), EmbedError>

Remove old shared CLIP rows after a deleted User’s private Index is purged. Photos may never have opened its lazy Indexer for this User, so account deletion calls this fixed-file cleanup directly (#435).

Source: crates/calternal-embed/src/clip_store.rs:39