docs / build with the engine

Behavior & lifecycle

Limits, workers, caches, results, privacy.

Cleanup Engine is a public beta. These guides are generated from the SDK repository at v0.2.0-beta.1; file links open it on GitHub.

Public request/result values are independent of MLX. PomvoxCleanup contains the local facade; CleanupCore owns results, guards and scheduling; CleanupPacks checks installed artifacts. Runtime/MLX is an optional Swift package so contract and transport users do not acquire GPU dependencies.

Limits and preparation

Requests permit 16,384 UTF-8 bytes of text; 64 vocabulary terms, 128 bytes each and 2,048 bytes combined; context up to 2,048 bytes; and at most 16 settings with 64-byte keys and 128-byte values. The frozen local baseline rejects any context/settings. Vocabulary cannot contain line breaks. Unsupported configuration fails before admission.

Preparation hashes local artifacts in chunks, loads using the directory-only model/tokenizer APIs, performs sampling-free prefix prefill, then generates a warmup. No fallback to a different model is possible. The preview only enables the optimized runtime for its exact supported artifact hashes. Opening fails if prefix/warmup preparation cannot complete.

A pack's capabilities (styles, speculative switch, auxiliary generation, minimum resident memory, vocabulary limits, and the guard rules identity) are available before open as ValidatedPack.capabilities and after as Cleaner.capabilities, so a host can avoid offering a control the pack would refuse or ignore. Refusing unsupported settings at request time is unchanged. See schema 2.

Admission limit and output cap

The two limits are one pair. A host reads the admission side from the capability query; the output side is enforced by the runtime.

They are not yet consistent for a single request. Admission does not check that an acceptable output fits the cap. The lower length guard needs at least 0.30 × the input's characters (0.15 with a correction marker), up to about 4,900 characters at the admission limit. Whether that fits in 1,024 tokens depends on the tokenizer. A request whose output does not fit ends as .fallback(.tokenLimit) (or .timedOut) with the exact input bytes, but only after decoding until the cap or the deadline. Words are never lost, but model time is. The core package cannot reject these requests at admission: deciding "cannot fit" means turning a character floor into a token count, which needs the pack's tokenizer. The admission-side check belongs to the runtime and is tracked by #13. The cap value itself is unchanged. For transcripts, opt-in TranscriptCleanOptions.segmentation splits a turn longer than 1,000 bytes (by default) at sentence ends into requests whose output fits the cap, and reassembles one result with exact byte offsets (see speaker transcripts).

Worker and request lifetime

One worker runs per cleaner and at most two requests queue. MLX also takes a process-wide resident-model lease, so another cleaner cannot allocate a second model until the first closes. This is a conservative preview constraint, not shared model pooling or a system-wide GPU lock.

Each request has one continuation and one deadline timer. A timeout or cancellation resolves the waiter without a structured task group waiting for inference to stop. The backend receives cooperative cancellation. A still-running worker is quarantined: new calls return unavailable, pending requests return unavailable, and no replacement worker starts. Late output is discarded. Resources are released only after the worker returns. close stops admission and can return before resource release if the worker is unresponsive.

The SDK cannot preempt a GPU kernel or provide hard real-time scheduling. A hung in-process worker can retain its device lease indefinitely. Process isolation is future work.

Cache isolation

Each runtime instance owns a single immutable prefix for its exact loaded artifact/tokenizer/prompt set. No cache survives a cleaner replacement. Every generation gets new mutable cache objects copied from that prefix. Vocabulary is rendered after the frozen prompt. The runtime retains one prefix for the current rendered vocabulary hint, rebuilding it when that hint changes. Configure .mlx(vocabulary: terms) to prepare the initial dictionary during open, and still supply those terms on every request. If the exact cached token prefix does not match, generation runs uncached and reports prefix-cache-not-used. There is no per-user cache dictionary. Instances are the scope boundary; multi-tenant serving and tenant cache pools are outside this local preview.

Hybrid recurrent layers legitimately report zero offsets and cannot trim rejected tokens. Prefix preparation requires counting layers to reach exactly the prefix length. Speculative decoding copies caches before verification, then restores and carries accepted tokens forward after a mismatch. Pass length remains five. The differential compares exact UTF-8, not visually equivalent Swift strings.

Results and privacy

Ordinary failures preserve input bytes. Empty input is unchanged without inference. Cancellation throws, including cancellation racing completion. The host must also prevent delivery from superseded sessions.

Requests may opt into deterministic post-transforms (transforms: [.spokenLayout], off by default). spokenLayout turns dictated "new line", "new paragraph" and "bullet" into layout, except where a modifier makes the phrase content ("a new line of products", "silver bullet"). It runs on accepted model output, so a cleaned or unchanged result's edits and status include it. It never touches a fallback: a fallback is still the exact input with no edits. A host that wants the commands honored on a fallback too calls SpokenLayout.apply or SpokenLayout.edits(from:) on that text itself. Provenance settings record an applied transform as transforms: spokenLayout. The cloud client applies transforms on the client and does not send them.

Edits are ordered, non-overlapping scalar-boundary ranges measured in original UTF-8 bytes. A single minimal contiguous replacement avoids quadratic diff work and preserves decomposed Unicode. It can include an unchanged interior span. No normalization occurs.

Provenance includes the pack manifest digest (which covers every artifact digest), model revision, runtime route and applied settings. Vocabulary is represented by a hash of its ordered JSON encoding; the diagnostic does not contain the words. Prefix/tokenizer identities are transitively pinned by the manifest. Hashes are identifiers, not anonymization promises.

Without an explicit deadline a request gets a bounded default budget (2 + 0.012·chars seconds, 5 to 60 s). Hosts that want the budget sized by a measured cost model can use CleanupBudgetPolicy: deadline(for:base:) returns a deadline never below the host's base and never above the policy ceiling (at most 60 s), or nil when even the ceiling cannot fit the estimated work, in which case the host keeps the exact input without calling the cleaner. measuredM1 holds the Apple M1 fit Pomvox measured on device (0.6 s + chars / 110 ch/s, × 1.4 headroom). Time the host spends outside the request, such as waiting for a model reload, is host timing and is not part of the policy.

Timings report preparation, queue, tokenization, prefill, decode/inference, validation, diff, total, and chosen budget. A stage not observed is nil. Preparation is separate from request total. The cloud client preserves server inference separately and replaces total with client-observed latency. Fallback timing never counts as successful model cleanup.

Local request code contains no network client, downloads, telemetry or content logging. Setup paths are private filesystem paths and should not be sent to external logging services. Local pack manifests are developer-trusted, hash-checked installations; hash validation does not authenticate a publisher. Signed remote pack distribution is future work. Installed files must remain immutable between validation and loading; hostile concurrent filesystem mutation is outside the preview trust model.

Transcripts and notes

A Transcript is an ordered list of SpeakerTurns (optional speaker label, text, optional span in seconds), at most 2,048 turns. clean(_ transcript:options:) on any Cleaning value sends one ordinary request per turn, in order, and returns one TurnResult per turn with the same speaker and span; the engine never merges, splits, reorders or relabels turns. Each turn is admitted, guarded, deadlined and falls back exactly as a dictation does. An optional overall deadline caps the whole call; once spent, remaining turns are .fallback(.timedOut) without inference. A turn over the admission limit is .fallback(.unavailable) without inference, not split or truncated, unless the host opts into segmentation, which splits long turns into ordinary requests and reassembles one result per turn. Turns the model never sees carry Provenance.skipped and one warning naming why (transcript-budget-exhausted, turn-exceeds-admission-limit, turn-budget-hopeless); every such turn is its exact bytes with no edits. Cancellation throws and delivers nothing. This is a batch call on the same one-request worker as live dictation. cleanIncrementally runs the same loop and yields each turn as it finishes. It can resume from a TranscriptCheckpoint of accepted results, which it refuses unless the transcript bytes, the output-affecting options, the pack and every accepted result match. Persistence stays with the host.

Transcript.attribute(words:segments:) is a pure function from timed words and speaker segments to turns; the nonempty input words appear in the output exactly once, in order. Notes.extract (module PomvoxNotes) runs no model: every note item is a verbatim sentence or list line of a cleaned turn with its UTF-8 byte range, participants are counts, and the whole TranscriptResult is carried in the value. See speaker transcripts and notes.

Integration additions

See gap review for dispositions and limitations, and the app integration prompt for host responsibilities. PackInstaller provides local verified installation with exclusive publication and APFS cloning where available. PackSource.validated reuses a process-local validation handle only while file identities remain unchanged. closeAndWait() waits for actual resource release and is cancellable; close() retains its nonblocking semantics. availability is a snapshot exposing quarantine, not an admission guarantee.

Completed runtime failures and guard rejections retain valid observed timings and warnings. Guard rejections include rejectedBy:<CleanupRejection.rawValue>. CleanupLogic.rulesVersion is checked against the manifest and reported as capabilities.guardRules, so a host can log which guard set produced a rejection. This repository owns the guard specification: GuardVectorTests pins every app guard vector to its rejection reason, so a change that moves a rejection to a different guard fails. Optional timing fields report prefix cache use, prompt/decode tokens and speculative rounds/drafted/accepted counts. Nil means unobserved. Timers do not wait for final worker statistics.

MLX buffer-pool clearing is opt-in through .mlx(vocabulary:clearBufferCache:) and is process-global when enabled. The default does not clear another workload's pool. Vocabulary bounds are the host's responsibility: select a stable bounded subset; invalid vocabulary throws rather than silently dropping requested spelling rules.