Billie Flow model analysis

The 12B audio model wasn’t the answer

I started with Gemma 12B because native audio sounded like the obvious route. On this memo it took 258.79s and drifted. MLX Whisper plus a small Qwen cleanup pass finished in about 4.30s and gave the better app default.

Expected Gemma 4 12B Audio 258.79s

One native-audio path, but far slower and affected by chunk-overlap drift.

Selected Whisper + Qwen 1.5B 4.30s

Separate recognition and cleanup, roughly 60× faster here and easier to inspect.

Clip: 35.32s
Recognition: MLX Whisper large-v3-turbo
Cleanup: Qwen2.5 1.5BMLX local small text
Evidence: 5 ASR branches, 70 cleanup runs

decision

The first app default is already clear enough

Recognition: MLX Whisper large-v3-turbo
Cleanup model: Qwen2.5 1.5BMLX local small text
Style: Light cleanup
Smoke fallback: MLX Whisper tinyTiny is a runner/smoke fallback only, not a quality fallback.

pick Default ASR: MLX Whisper large-v3-turbo

It remains the best first app default: much faster than the native-audio alternatives, coherent, and correct on LLM. It still needs vocabulary correction for Wispr Flow and Billie Flow.

pick Default cleanup: MLX local small text / light cleanup

Light cleanup is the right first app default because it removes dictation friction without pretending ASR mistakes did not happen.

watch Keep vocabulary correction explicit

The report should keep raw ASR visible. Wispr Flow, Billie Flow, LLM, and MacBook are the terms to bias or repair first.

avoid Do not ship Gemma, Voxtral, or Parakeet as defaults yet

They now run locally where public access allows, but they are slower and still miss the key vocabulary. Gemma also depends on the public google/gemma-4-12b-it checkpoint because the originally named google/gemma-4-12b-audio id does not exist.

branch map

One memo, five routes through the pipeline

The page stays diagram-first; transcripts sit behind the model rows.

Voice memo→16 kHz mono→ASR branch→Qwen cleanup→Vocabulary repair

default

MLX Whisper large-v3-turbo

3.68s

Chunking: whole-file
Vocabulary: 2 correct / 2 missed
Status: ran

Best default ASR result. It is fast, coherent, and hears LLM, but still needs vocabulary correction for Wispr Flow and Billie Flow.

smoke

MLX Whisper tiny

2.00s

Chunking: whole-file
Vocabulary: 1 correct / 3 missed
Status: ran

Good smoke test, not good enough for quality. It keeps the shape, but loses exactly the project vocabulary this app cares about.

lab

Gemma 4 12B Audio

258.79s

Chunking: fixed-25s-overlap-2s
Vocabulary: 2 correct / 2 missed
Status: ran

Completed, but not default-worthy. The public audio-capable Gemma 4 12B checkpoint ran, but chunk overlap introduced text drift and the path is far slower than Whisper.

lab

Voxtral Mini 3B

110.85s

Chunking: long-form
Vocabulary: 2 correct / 2 missed
Status: ran

High-readability lab candidate. It produces a clean transcript and gets LLM/MacBook, but the runtime is high and it collapses the project names.

lab

Parakeet TDT 0.6B v3

42.81s

Chunking: whole-file
Vocabulary: 2 correct / 2 missed
Status: ran

Strong lab candidate. It preserves filler and timing detail better than Whisper, but is much slower and still misses the product names.

ASR evidence

Click into the transcript only when the summary is not enough

MLX Whisper large-v3-turbo default / 3.68s 3/5

What it heard

Okay, so I am just going to ramble for a little bit and see how good the quality of this voice memo is, I suppose. So essentially, let's say that I wanted to build a project where we were making a local voice clone. So similar to Whisperflow, but not quite Whisperflow because we would call it Billy Flow, and the whole premise would be that it runs using a local LLM on my MacBook.

Read

Best default ASR result. It is fast, coherent, and hears LLM, but still needs vocabulary correction for Wispr Flow and Billie Flow.

Weaknesses

Normalizes Wispr Flow to Whisperflow
Writes Billie Flow as Billy Flow
Needs a vocabulary/context correction layer

MLX Whisper tiny smoke / 2.00s 2/5

What it heard

Okay, so I am just going to ramble for a little bit and see how good the quality of this voice memo is, I suppose. So essentially let's say that I wanted to build a project where we were making a local voice clone. So similar to whisper flow but not quite with flow because we would call it Billy Flow. The whole premise would be that it runs using a local LLL on my Macbook.

Read

Good smoke test, not good enough for quality. It keeps the shape, but loses exactly the project vocabulary this app cares about.

Weaknesses

Hears LLM as LLL
Splits Wispr Flow into the wrong phrase
Lower confidence around key terms

Gemma 4 12B Audio lab / 258.79s 3/5

What it heard

Okay, so I am just going to ramble for a little bit and um see how good the quality of this uh voice memo is, I suppose. Um so, essentially, let's say that I wanted to build a um project where we are making a local voice clone. Um so, similar to WhisperFlow but not quite WhisperFlow because exactly. I've been not quite with the flow because, we would call it, Billy flow, um, and the whole premise would be that it runs, um, using a local LLM on my MacBook.

Read

Completed, but not default-worthy. The public audio-capable Gemma 4 12B checkpoint ran, but chunk overlap introduced text drift and the path is far slower than Whisper.

Weaknesses

The originally named google/gemma-4-12b-audio Hub id does not exist
Chunk overlap produced drift around the Wispr Flow sentence
Too slow and too fragile for the first app default

Voxtral Mini 3B lab / 110.85s 3/5

What it heard

Okay, so I am just going to ramble for a little bit and see how good the quality of this voice memo is, I suppose. So essentially, let's say that I wanted to build a project where we were making a local voice clone. So similar to WhisperFlow, but not quite WhisperFlow, because we would call it BillyFlow. and the whole premise would be that it runs using a local LLM on my MacBook.

Read

High-readability lab candidate. It produces a clean transcript and gets LLM/MacBook, but the runtime is high and it collapses the project names.

Weaknesses

Writes Wispr Flow as WhisperFlow
Writes Billie Flow as BillyFlow
Much slower than the MLX Whisper default

Parakeet TDT 0.6B v3 lab / 42.81s 3/5

What it heard

Okay, so I am just gonna ramble for a little bit and um see how good the quality of this uh voice memo is I suppose. Um so essentially let's say that I wanted to build a um project where we were making a local voice clone. Um so similar to Whisperflow but not quite Whisperflow because we would call it Billy Flow. Um and the whole premise would be that it runs um using a local LLM on my MacBook.

Read

Strong lab candidate. It preserves filler and timing detail better than Whisper, but is much slower and still misses the product names.

Weaknesses

Normalizes Wispr Flow to Whisperflow
Writes Billie Flow as Billy Flow
Runtime is too slow for the first default on this memo

cleanup examples

Enough style evidence to judge the default, not seventy cards

MLX Whisper large-v3-turbo Light cleanup / MLX local small text

Best default-style candidate; keeps the important vocabulary visible.

fidelity 5/5 voice 4/5 invention 1/5

Okay, so I'm just going to ramble a little bit and see how good the quality of this voice memo is. Essentially, I want to build a local voice clone. It's similar to Wispr Flow, but not quite Wispr Flow. We'll call it Billie Flow. The premise is that it runs using a local LLM on my MacBook.

Whisperflow -> Wispr Flow (2), Billy Flow -> Billie Flow (1)

MLX Whisper large-v3-turbo Verbatim, context-corrected / MLX local small text

Best audit-style candidate; keeps the important vocabulary visible.

fidelity 5/5 voice 5/5 invention 1/5

Okay, so I am just going to ramble for a little bit and see how good the quality of this voice memo is. Essentially, I want to build a local voice clone. It would be similar to Wispr Flow, but not quite Wispr Flow because we would call it Billie Flow. The whole premise is that it would run using a local LLM on my MacBook.

Whisperflow -> Wispr Flow (2), Billy Flow -> Billie Flow (1)

MLX Whisper large-v3-turbo Notes / MLX local small text

Useful for scanning, less useful for fidelity review; keeps the important vocabulary visible.

fidelity 5/5 voice 3/5 invention 1/5

- Start rambling for a bit to assess voice memo quality - Goal: Build a local voice clone - Project name: Billie Flow - Premise: Local LLM on MacBook

Billy Flow -> Billie Flow (1)

MLX Whisper tiny Light cleanup / MLX local small text

Useful style-specific transform; source ASR errors still leak through; keeps the important vocabulary visible.

fidelity 4/5 voice 4/5 invention 2/5

Okay, so I'm just going to ramble a little bit and see how good the quality of this voice memo is. Essentially, I want to build a local voice clone. It would be similar to Wispr Flow but not quite with Billie Flow. The whole premise is to run it on my MacBook using LLM.

Whisper Flow -> Wispr Flow (1), Billy Flow -> Billie Flow (1), Macbook -> MacBook (1)

Gemma 4 12B Audio Light cleanup / MLX local small text

Useful style-specific transform; source ASR errors still leak through.

fidelity 3/5 voice 4/5 invention 2/5

Okay, so I'm just going to ramble a little bit and see how good the quality of this voice memo is. Essentially, I want to build a local voice clone. It's similar to Wispr Flow, but not quite like that. I've been working on it, and the whole premise is to run it on my MacBook.

WhisperFlow -> Wispr Flow (1)

vocabulary

The product names are the real test

Model	Wispr Flow	Billie Flow	LLM	MacBook
MLX Whisper large-v3-turbo	Whisperflow	Billy Flow	correct	correct
MLX Whisper tiny	whisper flow, with flow	Billy Flow	LLL	correct
Gemma 4 12B Audio	WhisperFlow, with the flow	Billy flow	correct	correct
Voxtral Mini 3B	WhisperFlow	BillyFlow	correct	correct
Parakeet TDT 0.6B v3	Whisperflow	Billy Flow	correct	correct

The app default should expose raw ASR, model cleanup, and final corrected text in debug mode so this repair layer stays visible.

method

Small enough to audit, not a benchmark claim

The source was a 35.3 second Voice Memos clip normalized to 16 kHz mono.
ASR and cleanup were evaluated as separate stages so polished text could not hide transcription errors.
Gemma, Voxtral, and Parakeet are useful lab evidence, but their runtimes and vocabulary misses keep them out of the first default.
Raw runner files are local lab evidence and are not embedded in this public artifact.

Setup notes kept visible

google/gemma-4-12b-audio: The originally configured Gemma audio Hub id is not a valid public model identifier. The run used google/gemma-4-12b-it instead and recorded that warning on the ASR result.
google/gemma-3n-E4B-it: Gemma 3n E4B is gated and was not available for this run.
MLX local strong text: mlx-lm 0.31.3 cannot load model_type gemma4_unified.; The cached Gemma 4 text checkpoint was not used as the strong cleanup model.