React · TypeScript · Browser AI2026 · 8 months · Product Engineer — AI pipeline, frontend, architecture

D-Mentalist

A real-time scripture detection and projection system that transcribes live speech in-browser and pushes detected verses to a second screen.

Overview

The product

D-Mentalist listens to spoken audio through the microphone and runs Whisper entirely in the browser via Transformers.js and ONNX Runtime — no server, no external API. Transcribed speech is matched against a verse database, and detected references roll into an operator console for review, staging, and projection to external displays over a BroadcastChannel. A dual-tier Web Worker architecture lets it choose a fast CPU path or a high-accuracy WebGPU tier.

100% in-browser

Transcription

CPU + WebGPU

Tiers

Second-screen ready

Projection

Firebase

Storage

The problem

Worship tech shouldn't need a rack of servers

Live scripture environments needed a display system that could turn spoken references into an on-screen verse projection in real time — but existing tools required server-side transcription, cloud API calls, internet dependence, and a camera-focused operator.

  • Transcription depended on remote APIs — slow, costly, privacy-heavy
  • No reliable pipeline from speech to verified verse detection
  • Operators needed a second-screen projection view with no extra infra

The solution

Server-less AI that runs in a browser tab

I designed a dual-tier Web Worker engine: whisper-tiny on CPU/Web Audio for low-latency detection, and whisper-large turbo on WebGPU for the overall accuracy. Spoken audio from the microphone streams into a Web Worker, is transcribed locally, then matched against the verse database with reverse lookup and semantic similarity. The operator reviews detections in a console and broadcasts the chosen passage live to a projection screen.

  • Fully in-browser transcription — zero-cloud service, zero external API calls
  • Dual-tier AI: a fast CPU tier and a high-accuracy WebGPU tier
  • Console + BroadcastChannel lets operators stage, queue, and push verses to a second screen

Architecture

The system behind D-Mentalist

The product is a pipeline: capture, transcribe, detect, review, project. Each stage is its own module so a stage can be swapped or audited independently.

  1. 01

    Operator Console

    React 19 · TanStack Router

    Mic control, preview, queue board, and live push to projection screens.

  2. 02

    Audio Capture

    Web Audio API

    Microphone input streamed into the engine — nothing leaves the device.

  3. 03

    Transcription Engine

    Transformers.js · ONNX Runtime

    Whisper models run in Web Workers on CPU/WebGPU, no server needed.

  4. 04

    Reference Detection

    Verse DB · Semantic similarity

    Transcribed text matched against a verse database via reverse lookup.

  5. 05

    Staging & Review

    Zustand · TanStack Query

    Detected verses land in a queue for operator review and staging.

  6. 06

    Projection Screen

    BroadcastChannel

    Approved verses broadcast to a second monitor or external display.

Tech stack

Tools that earned their place

Frontend

  • React 19
  • TypeScript
  • Vite
  • Tailwind CSS v4

Routing

  • TanStack Router
  • TanStack Query
  • Zustand

AI

  • Transformers.js
  • ONNX Runtime Web
  • Whisper

Science

  • BroadcastChannel
  • Web Audio API
  • Web Workers

Backend

  • Firebase

Features

What it does, and why it matters

01

In-browser AI

Whisper and embedding models run entirely on-device — no cloud calls.

02

Real-time verse detection

Detect scripture references live from spoken audio with semantic fallback.

03

Operator console

Preview, stage, queue, and push verses with a dashboard built for live events.

04

Second-screen projection

Approved verses broadcast to external displays via BroadcastChannel.

05

Dual-tier AI engine

A fast CPU tier and a high-accuracy WebGPU tier of Whisper choose themselves.

Challenges

What nearly broke the system

No server for transcription

I ran Whisper in the web browser with ONNX Runtime and Web Workers so audio never leaves the device — a real differentiator for privacy-sensitive environments.

Battery budget on a phone

Whisper is heavy for edge devices. I added a Tier-A CPU pipeline for fast detection and a Tier-B WebGPU fallback, so latency and battery stay in the operator's control.

Accuracy without hallucination

It's about validation, not guesswork. I combined reverse lookup plus semantic embeddings so detected references are confirmed in the console before anything is shown.

Lessons learned

What shipping taught me

Browser-based AI removes all the infra — and all the latency.

Two inference tiers beat one model: fast for live flow, accurate for the big moments.

A human-in-the-loop (operator reviewing) turns raw transcription into trustworthy output.

Impact

Why this actually matters

D-Mentalist removes every server from the loop — capture, transcription, detection, review, and projection all run on-device or between browser tabs, making live scripture environments faster, cheaper, and fully private.

  • Zero cloud calls means zero per-minute transcription cost and zero privacy risk
  • Dual-tier inference keeps the live flow fast on CPU and accurate on WebGPU
  • Operator review turns raw transcription into verified, projection-ready verses

Gallery

Inside the product

Operator console

Mic control, live queue, and push-to-project.

Projection screen

Verses streamed live to external displays.

AI engine

Whisper tiers running fully on-device.