calternal_notes_core::nlp

NLP composer — deterministic, Foundation-only logic ported from the Swift reference (ComposerNLPPipeline.run + JournalNLP.parse + ComposerModeClassifier). Produces a ParsedEntry from one line of capture text.

What is ported

tokenizer, single-time reader, time-range / single-time / duration recognizers, tag extraction (tokenizer/time/recognizers/tags modules).
date resolution (dates): relative + explicit dates resolved to a concrete YYYY-MM-DD via a from-scratch civil-date helper (civil), plus a day-first DD/MM slash-date ENHANCEMENT.
temporal anchors (anchors): since X / until X → resolved days, standalone duration → minutes.
mode classifier (classifier): event / task / note.
Task capture parsing, readable recurrence serialization and RRULE conversion (#430), task sigils, and the composer highlight-run projection (tasks, tasks_recurrence_md, tasks_sigils, and runs).
the pipeline ordering of ComposerNLPPipeline.run (anchors → dates → time-range / single-time + duration → standalone duration → tags → residual title → same-day default) and the byte-stable residual title.

What is intentionally NOT ported

The Apple FoundationModels LLM refiner (FoundationModelsRefiner, FMRefiner). We expose a clean trait seam (Refiner) instead.
The Swift confidence/diagnostics enums: the wire ParsedEntry carries resolved fields, not the review-routing band. The pipeline still computes the same claims so resolution + residual title match the corpus.

The deterministic pipeline is complete; the optional model refinement seam remains host-owned.

Deviations from Swift (documented, fidelity-justified)

Title casing: Swift cleanResidual capitalizes the first letter for display. The contract anchors keep user casing verbatim ("shipped v1 #work" -> title:'shipped v1'), so the core preserves casing and leaves capitalization to the host. The journal corpus asserts titles case-insensitively, so this does not break fidelity.
Tag namespacing: the parser applies the reference policy after mode classification: event tags use area/, task tags use task/, note tags stay literal, and explicit namespaces pass through unchanged.
Connector strip (orphaned_connectors): Swift cleanResidual strips a leading-stopword list from the head of the title string. That position-based strip caused two user-confirmed data-loss bugs: the list included the/i/was ("The Leftovers" → "Leftovers"), and even after pruning to genuine connectors, a title legitimately starting with connectors lost them ("on and off working on calternal.js 22:00 - 23:00" → "off working …"). The core instead strips a connector (and/at/from/where/to/in/for/of/on) ONLY when it was orphaned by an adjacent recognizer-consumed span in the token stream — position in the title never matters. Deliberate, correct divergence.
Tag stripping in titles: matches iOS ComposerNLPPipeline — ALL tag tokens are stripped from the residual title (no leaf word is kept). For example #area/work between title words is removed entirely; only the surrounding non-tag words remain in the title.

Source: crates/calternal-notes-core/src/nlp/mod.rs

Modules

Module	Summary
`task_types`	Value types returned by the task NLP extractors.
`tasks_recurrence_md`	Canonical serializer and deserializer for `TaskRecurrence`.
`tasks_sigils`	Sigil-to-tag projection for task inputs, ported from iOS `TasksNLP.swift` lines 160-161 (`@context` / `+project` handling).
`runs`	Composer highlight runs — UTF-16 spans for the mirror-overlay.

Re-exports

Structs

CivilDate

pub struct CivilDate

A civil (proleptic Gregorian) calendar date. Times are not modeled — the Swift recognizers all resolve to start-of-day, so a bare Y-M-D is exact.

Serde derives are added here (not in civil.rs originally) so that types in task_types which embed CivilDate can serialize to JS via serde-wasm-bindgen without wrapping the date in a string. The civil module stays chrono-free; these derives are the only serde dependency it adds.

Fields

pub year: i32
pub month: u32
pub day: u32

Implements: Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize

CivilDate::to_ymd

pub fn to_ymd(self) -> String

YYYY-MM-DD, zero-padded.

CivilDate::to_days

pub fn to_days(self) -> i64

Day-number since the civil epoch (1970-01-01 == 0). Hinnant’s days_from_civil. Valid for any proleptic-Gregorian date.

CivilDate::from_days

pub fn from_days(z: i64) -> CivilDate

Inverse of to_days. Hinnant’s civil_from_days.

CivilDate::add_days

pub fn add_days(self, delta: i64) -> CivilDate

Add delta days (may be negative).

CivilDate::weekday

pub fn weekday(self) -> u32

Weekday as a Foundation Calendar component: 1 == Sunday .. 7 == Saturday. Matches calendar.component(.weekday, from:) so the ported weekday arithmetic is byte-identical to Swift.

CivilDate::add_months

pub fn add_months(self, delta: i32) -> CivilDate

Add delta calendar months, clamping the day to the target month’s length (Foundation byAdding:.month keeps the day-of-month, but for the corpus inputs the day always exists in the target month). We clamp defensively so e.g. Jan 31 - 1 month doesn’t overflow.

Source: crates/calternal-notes-core/src/nlp/civil.rs:25

ModeClassification

pub struct ModeClassification

classifyModeDetailed result (#27 T42, spec C §D.2) — the ONE ranking source: mode is computed by the SAME classify()/score_with_reasons() pass ParsedEntry.mode uses, plus confidence/margin/reasons so the TS auto-switch reducer (composerMode.svelte.ts’s decide(), T43) applies pure hysteresis POLICY without ever re-scoring or re-ranking anything itself (spec §K: “reducer re-ranks independently” is the iOS bug this shape exists to make impossible by construction).

Fields

pub mode: String: "event" | "task" | "note".
pub confidence: String: "none" (auto-detect gate failed, mode is tab_default), "medium" (gate cleared), or "high" (gate cleared AND best >= HIGH_SCORE).
pub margin: i32: best score - second score. Computed unconditionally (even when the gate fails) — the §M boundary-matrix tests exercise sub-gate margins too, so this is a raw diagnostic value, not itself gated.
pub reasons: Vec<String>: One entry per weight the scorer applied, in a fixed reading order (event, then task, then note — see score_with_reasons‘s doc for why this differs from task_signals’ internal token-consumption order). Tests assert these by STABLE PREFIX (e.g. "task +10:"), never full- string equality — the iOS “source-string-pin” anti-pattern this design bans (spec §K).

Implements: Debug, Clone, PartialEq, Eq

Source: crates/calternal-notes-core/src/nlp/classifier.rs:60

Enums

ComposerMode

pub enum ComposerMode

Mode used to normalize composer tags.

Variants

Event
Task
Note

Implements: Debug, Clone, Copy, PartialEq, Eq

ComposerMode::from_wire

pub fn from_wire(mode: &str) -> Self

No doc comment.

Source: crates/calternal-notes-core/src/nlp/mod.rs:107

Traits

Refiner

pub trait Refiner

Seam for an optional, non-deterministic refinement pass (e.g. an LLM). The deterministic pipeline never calls this; a host wires it in where it wants review-only suggestions. Mirrors the Swift FMRefiner protocol; the implementation (FoundationModelsRefiner) is deliberately not ported.

Refiner::refine

fn refine(&self, text: &str, deterministic: &ParsedEntry) -> ParsedEntry;

Refine a deterministic draft. Implementations must treat the result as review-only and never auto-commit.

Source: crates/calternal-notes-core/src/nlp/mod.rs:162

Functions

classify_mode_detailed

pub fn classify_mode_detailed(text: &str, now: &str, tab_default: &str) -> ModeClassification

classifyModeDetailed entry point (#27 T42, spec C §D.2): re-derive the SAME ParsedInternal parse_entry_input computes (so the classifier sees identical journal signals) and run classifier::classify_detailed against an explicit tab_default — unlike parse_entry_input, which always hard-codes "event" (see its own comment), this fn accepts an arbitrary tab default so a future Tasks/Notes tab (or a test corpus) can seed a different baseline without a second code path. The qualified equivalence this unlocks — parse_entry_input(s, now).mode == classify_mode_detailed(s, now, "event").mode — is the ONE ranking source invariant pinned by the corpus property test (contracts.md §5).

now unparseable falls back to the epoch reference, mirroring parse_entry_input’s own None-branch (date-dependent task/event signals simply don’t fire — the conservative fallback).

Source: crates/calternal-notes-core/src/nlp/mod.rs:284

extract_tags

pub fn extract_tags(text: &str) -> Vec<ExtractedTag>

Extract deduped tags from text. The returned bodies have the leading #/## stripped (e.g. #area/work -> area/work). Order is encounter order; dedup is case-insensitive on the body, first occurrence wins (its had_double_hash flag is the one kept).

Source: crates/calternal-notes-core/src/nlp/tags.rs:110

format_date_token

pub fn format_date_token(date: &str, now: &str) -> String

Format a resolved calendar date into a composer-splicable date token (#27 T45, spec C §B.4): day-first "DD/MM" when date’s year matches now’s reference year, or year-qualified "DD/MM/YYYY" otherwise. Reuses the SAME day-first slash-date grammar find_slash_ddmm already reads back — including its existing optional /YYYY group — so no recognizer extension was needed for the year-qualified form, unlike the plan’s contingency (“if the recognizer can’t parse a year-qualified form, extending it is part of this task”): find_slash_ddmm accepted DD/MM/YYYY before this task existed (see its own doc comment/tests), so this fn only had to emit a shape it already knew how to read.

Marker-agnostic like time::format_time_token: for a due/scheduled/ start/anchor run, the CALLER wraps this bare date with the run’s preserved role marker ("{marker} {format_date_token(...)}", spec §B.4’s role-preservation rule) — this fn only ever emits the date itself.

date/now are YYYY-MM-DD (a longer ISO datetime also works — only the leading date matters, civil::parse_reference_date’s existing contract). An unparseable date degrades to the raw string verbatim (defensive only: every real caller passes an already-core-resolved date value, so this path is unreachable in practice but must not panic).

Source: crates/calternal-notes-core/src/nlp/dates.rs:460

format_time_token

pub fn format_time_token(start: &str, end: Option<&str>) -> String

Format a (start, optional end) "HH:MM" pair into a composer-splicable time token (#27 T45, spec C §B.4): "09:30" for a single time, or "09:30-10:15" for a range. Marker-agnostic — unlike a date/anchor run (see dates::format_date_token’s doc), a time run carries no role marker to preserve, so this is a plain concatenation with a bare - separator (already one of find_time_range’s accepted separators, no surrounding spaces needed — matches how bare-number ranges like "9-10" already round-trip in the existing corpus). Pinned by a round-trip property test: parse_entry_input(format_time_token(s, e) + " x", now) must recover the EXACT (s, e) pair, including midnight ("00:00") and a forward range — this is what makes the splice safe to feed straight back into the recognizer rather than a hand-built string it might read back differently. start/end are trusted to already be zero-padded "HH:MM" (every real caller is a native <input type="time"> value or a parsed.start/end the core itself resolved) — this fn does no validation, matching the “marker-agnostic, formats only” scope the spec draws around it.

Source: crates/calternal-notes-core/src/nlp/time.rs:208

normalize_tag

pub fn normalize_tag(raw: &str, had_double_hash: bool, mode: ComposerMode) -> String

Apply mode-aware namespacing to an NFC Tag value. Double-hash tags and explicit namespaces stay literal so a caller’s chosen scope is preserved.

Source: crates/calternal-notes-core/src/nlp/mod.rs:125

normalize_tag_value

pub fn normalize_tag_value(value: &str) -> String

Canonical spelling for a Tag value. Existing frontmatter stays untouched; new parsed and indexed values use NFC so composed spellings compare equal.

Source: crates/calternal-notes-core/src/nlp/tags.rs:85

parse_entry_input

pub fn parse_entry_input(input: &str, now: &str) -> ParsedEntry

Parse one line of capture text into a ParsedEntry.

now is an ISO datetime string (e.g. 2026-06-24T12:00:00Z); its calendar date is the reference for relative-date resolution. The clock/zone are ignored (the Swift pipeline resolves against start-of-day of the reference).

Source: crates/calternal-notes-core/src/nlp/mod.rs:195

parse_task_input

pub fn parse_task_input(text: &str, now: &str) -> ParsedTaskInput

Parse one line of task capture text into a ParsedTaskInput.

now is an ISO date/datetime string (e.g. "2026-06-30" or "2026-06-30T09:00:00Z"); only the calendar date prefix is used as the reference for relative-date resolution. If now cannot be parsed, date fields that need a reference are left as None — matching the fallback behaviour of parse_entry_input.

The title is capitalized (first letter upper-cased) to match the Swift cleanResidual display convention that the task composer surfaces to users.

Source: crates/calternal-notes-core/src/nlp/tasks.rs:42

split_cross_midnight

pub fn split_cross_midnight(date: &str, start: &str, end: &str) -> Vec<DaySegment>

Split a cross-midnight range into two forward day-segments — the single source of truth shared by every host (web composer + CLI) so they agree on the boundary byte-for-byte.

A range like 23:00–01:00 is reversed within one day (start > end); per the product spec it is ONE activity that crosses midnight, so it becomes TWO forward events on consecutive days:

day N: start–23:59
day N + 1: 00:00–end

We use 23:59 (not 24:00 — there is no 24:00 wall-clock) as day N’s end, matching the on-disk HH:MM contract and never producing a reversed range.

date is the base day (YYYY-MM-DD); start/end are the RAW HH:MM strings from a crossesMidnight ParsedEntry (so start > end). Pass them through verbatim — only the boundary times are synthesized. Callers must only invoke this when crossesMidnight is set. If date is somehow not a valid YYYY-MM-DD we fall back to the same date for day N+1 (defensive only; hosts always pass the composer’s validated base day).

Source: crates/calternal-notes-core/src/nlp/mod.rs:314

tag_name_parts

pub fn tag_name_parts(tag: &str) -> Vec<String>

Return the lower-case name parts of a Tag for tag:<name> search.

Keep the full namespaced Tag in the exact index field. Index each part separately so a query such as tag:fitness can find area/fitness without weakening a full query such as tag:area/fitness.

Source: crates/calternal-notes-core/src/nlp/tags.rs:143

valid_ymd

pub fn valid_ymd(year: i32, month: u32, day: u32) -> bool

Validate that (year, month, day) is a real calendar date. Mirrors the Swift NLPExplicitDateRecognizer.date(...) round-trip rejection of inputs Foundation would silently roll forward (e.g. Feb 31).

Source: crates/calternal-notes-core/src/nlp/civil.rs:133