docs / build with the engine

Local quick start

Open once, clean repeatedly, close when done.

Cleanup Engine is a public beta. These guides are generated from the SDK repository at v0.2.0-beta.1; file links open it on GitHub.

Quick start

1. Add the package

The root package has no MLX dependency. Local inference is distributed as the standalone PomvoxCleanupMLX package, generated from Runtime/MLX in this source repository. Add the public runtime repository in Xcode at exact version 0.2.0-beta.1, or declare:

// Package.swift dependency
.package(url: "https://github.com/pomvox/pomvox-cleanup-mlx.git", exact: "0.2.0-beta.1")

// Application target dependency
.product(name: "PomvoxCleanupMLX", package: "pomvox-cleanup-mlx")

Use an Xcode build to package the dependency's Metal shader resources. A plain swift build is insufficient to package the required shader library. The separate consumer demonstrates a working application build. The runtime pulls the matching core SDK version automatically; no submodule is required. Core/cloud-only consumers can instead add https://github.com/pomvox/pomvox-cleanup-engine.git at exact version 0.2.0-beta.1 and select PomvoxCleanup, PomvoxNotes, PomvoxDiarization or PomvoxCleanupCloud.

2. Prepare assets already on disk

The baseline requires the exact snapshot recorded in pack.json. Given that snapshot:

python3 scripts/prepare-local-pack.py /path/to/pinned/snapshot .local/simplewords-v3

Apps can use the native PackInstaller.install(snapshot:manifestData:destination:) with the trusted bundled manifest instead of invoking Python. Install outside cloud-synced folders, keep the pack immutable, and pass the returned handle to .validated(installedPack) when opening. Model access is separate from SDK installation: the pinned upstream model currently requires accepting access conditions.

The tool copies and verifies the seven pinned artifacts, refuses existing destinations and cleans up failed copies. It never downloads or modifies the source. The upstream model currently has an access gate; reproducible public acquisition and an archived license/notice record remain release requirements. See installed packs.

3. Open once, clean repeatedly, close when done

import Foundation
import PomvoxCleanupMLX // Exports the runtime-independent PomvoxCleanup API.

let cleaner = try await Cleaner.open(
    pack: .directory(URL(fileURLWithPath: "/absolute/path/to/installed/pack")),
    runtime: .mlx,
    policy: .local
)

let result = try await cleaner.clean(CleanupRequest(
    "um please send the pomvox report tomorrow",
    vocabulary: ["Pomvox"],
    deadline: .seconds(2)
))
print(result.text)
print(result.status)
await cleaner.close()

open verifies hashes, loads the model, prepares the prefix and warms Metal kernels. Missing or incompatible assets throw before a usable cleaner is returned. Keep the pack immutable while open. The host should close its cleaner on both success and error paths when retiring it; opening per request wastes preparation work.

The baseline supports English and bounded vocabulary. It rejects nonempty context or settings, including style. Unicode-safe edits do not imply multilingual model quality. Output guards are heuristics and cannot guarantee meaning preservation.

Outcomes and cancellation

Outcome	Meaning	Host action
`.cleaned`	Accepted output differs from input	Inspect/apply the result
`.unchanged`	Accepted output equals input, or input was empty	Preserve the result
`.fallback(reason)`	Cleanup did not produce an accepted result	Result contains the exact input bytes and zero edits
`CancellationError`	The caller canceled or its session was superseded	Do not insert text
Other thrown error	Invalid request, unsupported configuration or setup failure	Correct configuration or present an error

Fallback reasons include timeout, rejection, unavailability, queue saturation and token limits. The local cleaner runs one request and queues at most two. A canceled/timed-out worker keeps its resources until it returns; replacement work is refused while it remains unresponsive. close() stops admission immediately and may return before that worker releases resources. A GPU kernel cannot be forcibly interrupted by this SDK.

Explicit deadlines cover admitted queueing and request work. The default is min(60, max(5, 2 + 0.012 × characterCount)) seconds. Preparation is measured separately. See lifecycle and request limits.

Applying edits

let reconstructed = try TextEdit.applying(result.edits, to: originalTranscript)
// Reconstructed bytes equal result.text. Offsets refer to the original transcript.
for edit in result.edits {
    let swiftRange = try edit.range(in: originalTranscript)
    let textViewRange = try edit.utf16Range(in: originalTranscript)
    // Use the appropriate range for your host editor.
}

Edits use half-open UTF-8 byte offsets, not character offsets. Invalid or overlapping ranges throw. The local engine emits one minimal contiguous replacement; it may include an unchanged interior span and does not describe editorial intent. Keep the original transcript for mapping these ranges.

Speaker transcripts and notes

For recordings with more than one voice, the engine cleans a speaker-attributed transcript one turn per ordinary request and keeps the structure intact. The host supplies recognized words with timings and who-spoke-when segments; Transcript.attribute aligns them, cleaner.clean(transcript) cleans each turn through the same admission, guards and fallback, and PomvoxNotes extracts deterministic notes (action items, decisions, questions, list items, participants) as verbatim quotes with provenance, rendered to Markdown. No model writes the notes.

import PomvoxNotes // re-exports the transcript types

let transcript = Transcript.attribute(words: timedWords, segments: speakerSegments)
let cleaned = try await cleaner.clean(transcript, options: TranscriptCleanOptions(overallDeadline: .seconds(120)))
let markdown = NotesRenderer.markdown(Notes.extract(from: cleaned))

A turn the model never sees keeps its exact bytes and says why in a warning. Diarization itself needs audio and a model: the root package carries only the PomvoxDiarization contract and a verified diarizer pack format, and the optional Runtime/Diarization package (PomvoxDiarizationFluid) runs FluidAudio 0.15.5's offline pyannote community-1 pipeline on the Neural Engine from a pinned, hash-checked pack (packs/pyannote-community1-coreml), never downloading. See speaker transcripts and notes.

Try it from a browser

A browser cannot run the SDK, but it can talk to a local companion process that does. Examples/LocalServer serves the cleanup and diarization wire protocols on 127.0.0.1 over the real local runtimes, plus a demo endpoint for the whole speaker-notes pipeline, with a one-page web app: record in the browser, get who-said-what, cleaned turns and notes back.