MediaPipe YAMNet (521 environmental sound categories) top-8 classification of a microphone recording (MediaRecorder; stop -> automatic classification; recording indicator) or an uploaded audio file, rendered as a ranked list with the top prediction highlighted. Ships a justified block-local record->classify hook (use-audio-classifier.ts — imports @localmode/core classifyAudio + @localmode/mediapipe audioClassifier under the blocks carve-out; not promotable as a zero-@localmode registry hook). Fully on-device. Regrouped from ui/blocks/vision-lab (Audio tab) into the audio category.
Media4 registry deps · 3 packages
Long-text streaming synthesis (useStreamSpeech) with early playback that begins before synthesis completes, a live clause counter + current-clause display, speed control (0.5x–2.0x), pause/resume/stop transport, a WAV download (audio-{voiceId}-{timestamp}.wav), a one-shot single-take synthesis path, a 10,000-character limit with pre-flight validation, and one-click sample texts. Kokoro downloads only on the first synthesize/stream action. Fully on-device. Split from ui/blocks/audio-studio (Audiobook tab).
Media8 registry deps · 3 packages
Remove an image's background entirely on-device: SegFormer (Xenova/segformer-b0-finetuned-ade-512-512, ~15MB) segments the subject, the highest-scoring mask is composited as an alpha channel (promoted applyMaskToImage) to a transparent-background PNG shown on a checkerboard Original/Result comparison, with a processing overlay + cancel, best-mask confidence, Download PNG (promoted downloadDataUrl), clear/reset, and dismissible errors with a retry path. The model loads lazily on the first dropped image; nothing downloads on page load. Split out of the tabbed image-studio Remove BG tool (split-image-privacy).
Media6 registry deps · 3 packages
The flagship on-device chat: a 76-model multi-provider selector (transformers, WebLLM, wllama GGUF, LiteRT) with capability gating, vision attachments, DeepSeek-R1 reasoning display, semantic response caching, agent mode with tools, custom GGUF URLs, system prompt + parameter controls, and slash commands — everything runs in the browser.
AI22 registry deps · 7 packages
Fill-mask word prediction via ModernBERT (top-5) with [MASK] authoring, ranked predictions (token, confidence score, confidence label, score bar) and a substituted full-sentence preview, and click-to-apply-and-iterate. This capability is Transformers.js-only — there is no Chrome AI fill-mask — and the badge says so. No model bytes download until an explicit in-block action.
Other6 registry deps · 3 packages
Schema-validated structured-data extraction from free text over its OWN on-device WebGPU-only WebLLM model (default Qwen3-1.7B-q4f16_1-MLC): five zod templates (contact, event, review, recipe, job), useGenerateObject json mode with maxRetries 3 retry/self-correction and a visible attempts counter, and an artifacts canvas that docks the validated JSON into a sortable data table + a chart built from the extraction's REAL numeric data (documented empty-chart state otherwise). Ships a justified block-local WebLLM catalog slice (model-catalog.ts) + the five templates + deriveArtifacts (templates.ts). Models download only behind an explicit Load action. Split from ui/blocks/agent-structured-data (Data Extractor mode).
AI10 registry deps · 4 packages
A zero-download device capability report rendered entirely from browser APIs (useCapabilities / useStorageQuota / useAdaptiveBatchSize): hardware cores/memory/GPU name, six feature flags (WebGPU/WASM/SIMD/Threads/IndexedDB/Workers), storage used/quota/available, browser + OS line, a reference 1B-Q4_K_M readiness verdict, and adaptive batch size + reasoning. Composes device-badge, device-capability-grid, browser-compat-card, storage-meter, and adaptive-batch-card. Zero network and zero model bytes on mount. Split from ui/blocks/device-model-lab (device report section).
Other5 registry deps · 3 packages
A global on-device observability drawer: six tabbed surfaces (inference queues, event log, pipeline runs, model cache, device capabilities, per-collection VectorDB stats) over the @localmode/devtools bridge, plus a lazy framework-agnostic host — devtools code loads only on first open, closing keeps collectors running, power-off fully disables.
Overlays7 registry deps · 2 packages
Extractive question answering (Xenova/distilbert-base-cased-distilled-squad, grounding corpus context via engine.search topK 1, High/Medium/Low confidence tiers) over its OWN text/PDF corpus, plus Donut (Xenova/donut-base-finetuned-docvqa) document/invoice QA on an uploaded image (~800 MB, gated). CORE engine only — declares no @localmode/langchain or @localmode/wllama. Split from ui/blocks/knowledge-base (Ask tab QA lanes; absorbs qa-bot + invoice-qa).
Other6 registry deps · 5 packages
Union-find near-duplicate detection over cached CLIP image embeddings (no re-embedding) — a tunable threshold slider + Strict/Balanced/Relaxed presets, per-group average similarity via a cosine-similarity-meter, keep-first selection + bulk delete, and a cancellable scan that re-groups (never re-embeds) on threshold change. Own usePhotoLibrary instance + its own media-dropzone ingest. Split from ui/blocks/photo-search (Duplicates tab).
Media7 registry deps · 4 packages
A passphrase-locked, end-to-end encrypted item store driven by useEncryptedVault — AES-GCM ciphertext at rest in IndexedDB, the key derived from the passphrase in memory only — with encrypted notes and imported text documents rendered as lock-aware vault-item-cards (decrypt-on-view + delete), a lock-status badge, a manual Lock that clears the in-memory key, a locked state that survives reload, and a tamper-evident hash-chained audit log (createAuditLog) with in-UI chain verification and JSONL export. Web Crypto + IndexedDB only — no models download. The IndexedDB vault name is kept at 'privacy-vault' for storage-key stability across the category rename. Split out of the tabbed privacy-vault Vault tab (split-image-privacy).
Other5 registry deps · 3 packages
Browse 160,000+ HuggingFace GGUF models and the curated wllama catalog, inspect any file's metadata via a ~4KB HTTP Range read (checkGGUFBrowserCompatFromURL — never a model download), get a browser-compat verdict, and hand a model off to the chat block. Consumes the PROMOTED @localmode/wllama model-discovery utils (searchGGUFModels / listGGUFFiles / HFApiError) instead of a block-local hf-api.ts; keeps a justified block-local chat-handoff emitter (handoff.ts). Cursor pagination, typed search errors + retry, and an explicit Download-to-cache prefetch. No model bytes on load. Split from ui/blocks/device-model-lab (GGUF explorer section).
AI7 registry deps · 2 packages
Generate alt-text captions for images with ViT-GPT2 (Xenova/vit-gpt2-image-captioning, ~230MB) entirely on-device, accumulating results in a gallery (caption as the card label, native per-card delete) with copy-caption-to-clipboard + transient copied feedback, remove-single, Clear All, a captioned-count indicator, and a compact add-another dropzone once results exist. Validates JPEG/PNG/WebP/GIF up to 10MB; the model loads lazily on the first dropped image. Consumes none of the promoted image ops. Split out of the tabbed image-studio Caption tool (split-image-privacy).
Media3 registry deps · 4 packages
Upscale and restore photos with Swin2SR super-resolution running fully in the browser — a 2x lightweight default (Xenova/swin2SR-lightweight-x2-64), plus 4x classical (Xenova/swin2SR-classical-sr-x4-64) and a real-world Restore variant (Xenova/swin2SR-realworld-sr-x4-64-bsrgan-psnr), each behind its own lazily-loaded model with persistent per-mode results. Presents the result in an Original/Enhanced toggle with a model/scale/status info panel and an Enhanced Nx badge; Download Enhanced PNG (promoted imageResultToDataUrl/getImageDimensions/downloadDataUrl), upload-new, cancel with a first-run hint, and dismissible validation + inference errors. Nothing downloads until you drop an image. Split out of the tabbed image-studio Enhance tool (split-image-privacy).
Media5 registry deps · 3 packages
Text→image AND image→image search over one shared CLIP vector space — embed a free-text query or a reference image and rank the ingested photo library with a top-K slider (default 20), a visible min-similarity threshold (getDefaultThreshold ?? 0.2), and CLIP-scale score bars. Own usePhotoLibrary instance + its own media-dropzone ingest. Split from ui/blocks/photo-search (Search tab; absorbs cross-modal-search + product-search).
Media8 registry deps · 4 packages
Debounced (400 ms) automatic language detection across 110 languages (up to five candidates with name + ISO code + confidence) that runs once typed text reaches 10 characters with no button press, plus a two-text cosine-similarity comparison via the on-device MediaPipe text embedder. Fully on-device; models download only when typing crosses the auto-detect threshold or Compare is pressed. Regrouped from ui/blocks/vision-lab (Language tab) into a new standalone text category (roadmap D2 — NOT merged into text-insights).
Text3 registry deps · 3 packages
Real-time MediaPipe streaming trackers on live webcam video — one 4-mode block: Hands (21-point skeleton + handedness), Pose (33-point skeleton + person count), Face (478-point mesh + top-8 expression blendshapes behind a default-on toggle), and Gestures (recognized category with a human-readable label). Live FPS; exactly one tracker/WASM task runs at a time. Ships a justified block-local hot-path canvas util (draw-landmarks.ts — single consumer, imports only @localmode/core topology constants under the blocks carve-out). Fully on-device. Split from ui/blocks/vision-lab (Track tab).
Feedback6 registry deps · 4 packages
Open-mic streaming speech-to-text (useLiveTranscribe) with VAD-segmented utterances, plus a listen→plan→speak turn-taking assistant (useTurnTaker: Granite 4.0 350M plans, Kokoro speaks) selectable via a two-mode picker. VAD choice: the zero-download energy VAD (default) or a gated ~2MB Silero ONNX VAD. Stop calls dispose() so the microphone MediaStream is released; barge-in + interrupt supported. Owns its own STT selector, its own adapter-backed inference-device probe for the Granite planner, and holds the Silero provider block-local (the module-scoped Silero singleton in audio-studio/models.ts is eliminated). Fully on-device. Split from ui/blocks/audio-studio (Live tab).
Media6 registry deps · 3 packages
Audio upload OR pasted transcript → Transcribe → Summarize (DistilBART, maxLength 200 / minLength 60) → Extract action items (useGenerateObject on Granite 4.0 350M) with a visible three-step indicator, a cancel that preserves completed steps, action-item priority badges + completion toggles with a live X/Y count, a [No speech detected] short-circuit, and a structured dated .txt export. Owns its own STT selector (Whisper Tiny EN default + Moonshine-Base recommendation hint) and its own adapter-backed inference-device probe for the Granite LM. Fully on-device. Split from ui/blocks/audio-studio (Meeting tab).
Media5 registry deps · 3 packages
Ranked on-device model recommendations that compute their OWN device context via useCapabilities and rank useModelRecommendations / getModelRegistry over the in-memory registry with ZERO network: a 21-task grouped selector, ranked model-recommendation-card results with a device-context line + result-count + empty states, two-model comparison via model-comparison-panel (max 2, third replaces the older), and a collapsible custom-model registration form (11 ModelRegistryEntry fields, per-field validation, in-memory notice, registerModel() + refresh). No model download. Split from ui/blocks/device-model-lab (model advisor section).
Layout3 registry deps · 3 packages
Evaluate a selected classifier over a selected labeled dataset via useEvaluateModel, reporting accuracy plus macro precision/recall/F1, run duration, and a color-coded confusion matrix, with completed/total progress and cancel, JSON export, and clear-results. Keeps one justified local file (datasets.ts) holding the labeled evaluation datasets and the radio-card selector, too large to inline without harming readability (design D5). No model bytes download until an explicit in-block action.
Text6 registry deps · 4 packages
One-shot DETR (Xenova/detr-resnet-50) object detection on an uploaded image, the bundled sample, or a webcam-captured still — color-coded bounding boxes, a unique-label legend, an N-found count, a ranked scored list, in-flight download progress with cancel, clear-to-reset, and dismiss+retry — plus a live MediaPipe BlazeFace webcam face loop with a numeric face count and captured-still overlay. Fully on-device; models download only behind explicit actions. Split from ui/blocks/vision-lab (Detect tab).
Other8 registry deps · 5 packages
Editable-label-set categorization with Photo/Product presets, on-device library re-categorization with progress + cancel (no re-embedding), and a counted, single-select category-facet-list filter over the ingested photos. Own usePhotoLibrary instance + its own media-dropzone ingest (photos are zero-shot categorized as they embed). Split from ui/blocks/photo-search (Categories tab).
Media7 registry deps · 4 packages
Detect PII with on-device NER (Xenova/bert-base-NER, BIO prefixes stripped, low-confidence filtered), redact it with reverse-offset [<TYPE>] placeholders honoring per-type toggles plus a core redactPII() regex pass (emails/phones/SSNs/cards), render entity insights (stats bar + relationship graph), copy/export the redacted text, and demonstrate differential privacy over the redacted text's embedding (Xenova/all-MiniLM-L6-v2, ~23MB) via dpEmbeddingMiddleware + a createPrivacyBudget tracker with a DP-applied provenance badge. Models download only behind the explicit Scan action. Ships a block-local redaction.ts (pure entity-mapping/redaction/graph helpers) alongside the component — justified as >130 lines of block-specific logic, too large to inline. Split out of the tabbed privacy-vault Redact tab (split-image-privacy).
Other5 registry deps · 4 packages