Skip to content

Commit fd5a87b

Browse files
committed
feat(web): make transcribing and editing feel less like a black box
Two friction points from a UX pass: the transcribe flow had no way out once started (a wrong model pick or a slow download meant reloading the page) and no visibility into what pressing Transcribe would actually cost; the transcript editor could only fix a word that whisper already produced, one at a time, with no signal for which words were worth double-checking. Transcribe flow: a model-cache hint next to the select says whether the chosen model is already on disk or about to download N MB, before the button is pressed. A Cancel button aborts an in-flight download via AbortSignal, or -- since whisper.cpp blocks the worker's one thread and can't be cooperatively interrupted -- destroys and replaces the worker outright when cancelled mid-transcription. That path exposed a real bug: engine.destroy() never rejected the promises it left pending, so a cancel there would have hung forever; fixed to reject them the same way a crashed worker already does. The single-threaded-host notice moved off the footer, which nobody scrolls to, and onto an inline hint next to the controls it actually explains. Transcript editor: word confidence was already computed by the whisper adapter and carried on every Word, but never rendered -- low-confidence words are now dimmed with a warning underline. New insertWord/deleteWord core ops (mirroring the existing split/merge/moveBoundary contract: pure, no-op by reference at the edges) back new "+" insert affordances and a Ctrl/Cmd+Backspace delete gesture, and committing an edit via Enter or Tab now opens the next word automatically, so correcting a run of words no longer needs a double-click per word.
1 parent afe1ec9 commit fd5a87b

6 files changed

Lines changed: 465 additions & 39 deletions

File tree

‎site/main.js‎

Lines changed: 106 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,10 @@ const transcribeButton = el('button', 'action is-primary', 'Transcribe')
109109
transcribeButton.type = 'button'
110110
transcribeButton.disabled = true
111111

112+
const cancelButton = el('button', 'action is-danger', 'Cancel')
113+
cancelButton.type = 'button'
114+
cancelButton.hidden = true
115+
112116
const srtButton = el('button', 'action', 'Download SRT')
113117
srtButton.type = 'button'
114118
srtButton.disabled = true
@@ -127,39 +131,49 @@ progress.max = 100
127131
progress.hidden = true
128132

129133
const controls = el('div', 'controls')
130-
controls.append(modelLabel, modelSelect, transcribeButton, srtButton, vttButton)
134+
controls.append(modelLabel, modelSelect, transcribeButton, cancelButton, srtButton, vttButton)
135+
136+
// Whether the selected model is already on this machine, and whether this
137+
// host runs single- or multi-threaded -- both said once, next to the
138+
// controls they explain, rather than discovered after clicking Transcribe or
139+
// buried in a footer nobody scrolls to.
140+
const modelHint = el('p', 'model-hint')
141+
const speedHint = el(
142+
'p',
143+
'speed-hint',
144+
// Only the non-default case is worth a sentence -- the isolated case is
145+
// what everything else on the page already assumes.
146+
THREADS_AVAILABLE
147+
? ''
148+
: 'Running single-threaded on this host, so transcription will be ' +
149+
'noticeably slower than on a cross-origin-isolated deploy.',
150+
)
151+
const controlHints = el('div', 'control-hints')
152+
controlHints.append(modelHint, speedHint)
131153

132154
// Empty until a transcript exists (`.transcript-hint:empty` hides it), so the
133-
// two gestures nothing else on the page states -- double-click a word,
134-
// drag ⋮ -- are said once rather than left for hover to reveal by accident.
155+
// gestures nothing else on the page states -- double-click a word, tab
156+
// through corrections, insert or delete one, drag ⋮ -- are said once rather
157+
// than left for hover to reveal by accident.
135158
const transcriptHint = el('p', 'transcript-hint')
136159
const TRANSCRIPT_HINT_TEXT =
137-
'Double-click a word to fix it · drag ⋮ between segments to move where one ends.'
160+
'Double-click a word to fix it, Enter/Tab for the next · + inserts a word, ' +
161+
'Ctrl/Cmd+Backspace deletes one · drag ⋮ between segments to move where one ends.'
138162

139163
const transcriptEl = el('div', 'transcript')
140164

141-
const footer = el(
142-
'p',
143-
'footer',
144-
THREADS_AVAILABLE
145-
? 'Cross-origin isolated: threaded inference is available.'
146-
: 'Not cross-origin isolated, so SharedArrayBuffer is unavailable and ' +
147-
'transcription runs single-threaded. That is the deploy host, not ' +
148-
'the build.',
149-
)
150-
151165
app.append(
152166
heading,
153167
chooser,
154168
status,
155169
surface,
156170
audio,
171+
controlHints,
157172
controls,
158173
transcriptHint,
159174
transcriptEl,
160175
progress,
161176
engineStatus,
162-
footer,
163177
)
164178

165179
const waveform = createWaveform(canvas)
@@ -176,10 +190,16 @@ const transcriptView = createTranscriptView({
176190

177191
// Substituted by the build, which bundles the worker first so that this name --
178192
// which carries a content hash -- exists to substitute. See types/build.d.ts.
179-
const engine = createEngineClient(new URL(__WHISPER_WORKER__, document.baseURI).href)
193+
const workerUrl = new URL(__WHISPER_WORKER__, document.baseURI).href
180194

181195
// --- State -----------------------------------------------------------------
182196

197+
// A `let`, not a `const`: cancelling mid-transcribe replaces it with a fresh
198+
// worker (see transcribe()'s catch/cancel handling below) rather than trying
199+
// to make an already-running whisper.cpp call give up the main thread it is
200+
// blocking.
201+
let engine = createEngineClient(workerUrl)
202+
183203
/** The object URL of the file currently loaded, released when the next replaces it. @type {string | null} */
184204
let currentUrl = null
185205
/** @type {AudioBuffer | null} */
@@ -212,6 +232,33 @@ function updateExportState() {
212232
transcriptHint.textContent = words > 0 ? TRANSCRIPT_HINT_TEXT : ''
213233
}
214234

235+
/**
236+
* Say, before someone commits to it, whether pressing Transcribe with the
237+
* currently-selected model starts a real download or not -- the option
238+
* label already states its size, but not whether that size has already been
239+
* paid.
240+
*
241+
* Stamped with the model id it was asked about and re-checked against the
242+
* select's current value before applying: `isModelCached` is async, and a
243+
* fast second change while the first check is still in flight must not have
244+
* its answer overwritten by the first one arriving late.
245+
*/
246+
async function refreshModelHint() {
247+
const id = modelSelect.value
248+
const model = MODELS[/** @type {keyof typeof MODELS} */ (id)]
249+
if (!model) return
250+
251+
const cached = await isModelCached(model.url, { caches })
252+
if (modelSelect.value !== id) return // superseded by a later selection
253+
254+
modelHint.textContent = cached
255+
? `${model.label} is already downloaded.`
256+
: `${model.label} will download ${megabytes(model.bytes)}.`
257+
}
258+
259+
modelSelect.addEventListener('change', () => void refreshModelHint())
260+
void refreshModelHint()
261+
215262
// Ctrl/Cmd+Z to undo, Ctrl/Cmd+Shift+Z or Ctrl/Cmd+Y to redo. Attached to
216263
// window rather than the transcript container, because a merge or split
217264
// click may have just had render() replace the element that had focus --
@@ -304,15 +351,41 @@ surface.addEventListener('drop', (event) => {
304351

305352
transcribeButton.addEventListener('click', () => void transcribe())
306353

354+
// The in-flight run's cancellation, if any -- created fresh by transcribe()
355+
// each time, since an AbortController can't be un-aborted for a second run.
356+
/** @type {AbortController | null} */
357+
let cancelling = null
358+
let cancelled = false
359+
360+
cancelButton.addEventListener('click', () => {
361+
cancelled = true
362+
cancelling?.abort()
363+
// There is no cooperative cancel in the worker protocol -- whisper.cpp
364+
// blocks the worker's one thread for the length of a transcribe() call, so
365+
// the only way to actually stop one already running is to end the worker
366+
// and start over. Doing this unconditionally (even mid-download, when
367+
// nothing has been loaded into this engine yet) is simpler than tracking
368+
// which phase is active, and costs one extra Worker construction that
369+
// does not itself load wasm or a model.
370+
engine.destroy()
371+
engine = createEngineClient(workerUrl)
372+
loadedModel = null
373+
})
374+
307375
async function transcribe() {
308376
if (!currentBuffer) return
309377

310378
const id = modelSelect.value
311379
const model = MODELS[/** @type {keyof typeof MODELS} */ (id)]
312380
if (!model) return
313381

382+
const controller = new AbortController()
383+
cancelling = controller
384+
cancelled = false
385+
314386
transcribeButton.disabled = true
315387
modelSelect.disabled = true
388+
cancelButton.hidden = false
316389
progress.hidden = false
317390
progress.removeAttribute('value')
318391
engineStatus.classList.remove('is-error')
@@ -330,6 +403,7 @@ async function transcribe() {
330403
const bytes = await downloadModel(model.url, {
331404
fetch: globalThis.fetch.bind(globalThis),
332405
caches,
406+
signal: controller.signal,
333407
onProgress: ({ loaded, total, cached: fromCache }) => {
334408
if (fromCache) return
335409
engineStatus.textContent = total
@@ -369,12 +443,26 @@ async function transcribe() {
369443
updateExportState()
370444
transcriptView.render(transcript)
371445
} catch (error) {
372-
engineStatus.textContent = message(error)
373-
engineStatus.classList.add('is-error')
446+
// Both the abort()ed fetch and the destroy()ed worker reject with their
447+
// own, unrelated errors -- `cancelled` is the one flag set by exactly one
448+
// cause, the Cancel button, so it is what distinguishes "asked for this"
449+
// from "actually failed" regardless of which phase was interrupted.
450+
if (cancelled) {
451+
engineStatus.textContent = 'Cancelled.'
452+
} else {
453+
engineStatus.textContent = message(error)
454+
engineStatus.classList.add('is-error')
455+
}
374456
} finally {
457+
cancelling = null
375458
progress.hidden = true
459+
cancelButton.hidden = true
376460
transcribeButton.disabled = false
377461
modelSelect.disabled = false
462+
// A cancelled download leaves nothing cached (downloadModel only stores
463+
// it on success), so this correctly falls back to "will download" rather
464+
// than assuming the run that just ended left the model behind.
465+
void refreshModelHint()
378466
}
379467
}
380468

‎site/styles.css‎

Lines changed: 85 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,55 @@ noscript p {
290290
background: transparent;
291291
}
292292

293+
/* Low-confidence words: dimmed, and given a warning-coloured underline
294+
instead of the neutral one every word gets at rest. Absent confidence
295+
(whisper didn't report one) draws with neither -- an unknown is not a low
296+
score. */
297+
.transcript-word.is-low-confidence {
298+
opacity: 0.6;
299+
border-bottom-color: color-mix(in srgb, var(--error) 55%, transparent);
300+
}
301+
302+
/* The insert control: a slot before every word and after the last one --
303+
unlike split, it is valid at a segment's very first and very last gap too,
304+
since a new word can be added at either edge. Same rest-state idiom as
305+
.transcript-split: invisible until found. */
306+
.transcript-insert {
307+
display: inline-block;
308+
position: relative;
309+
width: 0.75rem;
310+
height: 1em;
311+
vertical-align: middle;
312+
margin: 0 -0.375rem;
313+
padding: 0;
314+
border: 0;
315+
border-radius: 2px;
316+
background: transparent;
317+
cursor: text;
318+
transition: background-color 140ms ease-out;
319+
}
320+
321+
.transcript-insert::after {
322+
content: '+';
323+
position: absolute;
324+
top: 50%;
325+
left: 50%;
326+
transform: translate(-50%, -50%);
327+
font-size: 0.7em;
328+
line-height: 1;
329+
color: color-mix(in srgb, var(--muted) 45%, transparent);
330+
}
331+
332+
.transcript-insert:hover,
333+
.transcript-insert:focus-visible {
334+
background: color-mix(in srgb, var(--accent) 35%, transparent);
335+
}
336+
337+
.transcript-insert:hover::after,
338+
.transcript-insert:focus-visible::after {
339+
color: var(--fg);
340+
}
341+
293342
/* Between segments: the drag handle and the merge control share this row,
294343
the way controls sharing a boundary in a timeline editor usually do -- one
295344
strip of affordances rather than two competing for the same pixels. */
@@ -337,7 +386,8 @@ noscript p {
337386

338387
.transcript-boundary-handle:focus-visible,
339388
.transcript-merge:focus-visible,
340-
.transcript-split:focus-visible {
389+
.transcript-split:focus-visible,
390+
.transcript-insert:focus-visible {
341391
outline: 2px solid var(--accent);
342392
outline-offset: 1px;
343393
}
@@ -394,6 +444,20 @@ noscript p {
394444
color: var(--bg);
395445
}
396446

447+
/* Cancel: the same --error token .status.is-error already uses, so a
448+
cancellable run and a failed one read as related states, not two
449+
unrelated colours invented for this one button. */
450+
.action.is-danger {
451+
border-color: var(--error);
452+
color: var(--error);
453+
}
454+
455+
.action.is-danger:hover:not(:disabled) {
456+
background: color-mix(in srgb, var(--error) 12%, transparent);
457+
border-color: var(--error);
458+
color: var(--error);
459+
}
460+
397461
/* Disabled controls are dimmed rather than hidden. Every one of them is
398462
disabled for a reason the status line states -- no file open, no transcript
399463
yet -- and a control that vanishes leaves that sentence answering a question
@@ -419,15 +483,26 @@ noscript p {
419483
margin-bottom: var(--sp-1);
420484
}
421485

422-
/* The threading readout. Deliberately quiet -- it is here to be checked, not
423-
read. It is also the only externally visible fact about the deploy host, so
424-
it survives the placeholder it arrived with. */
425-
.footer {
426-
margin-top: var(--sp-4);
427-
padding-top: var(--sp-2);
428-
border-top: 1px solid var(--line);
429-
font-family: var(--mono);
486+
/* Two short lines that belong next to the controls they explain: whether the
487+
selected model needs downloading, and whether this host runs single- or
488+
multi-threaded. Grouped so both can be visible at once (a plausible first
489+
visit: an uncached model on a non-isolated host) without each fighting the
490+
controls row for its own line. */
491+
.control-hints {
492+
display: flex;
493+
flex-direction: column;
494+
gap: var(--sp-1);
495+
margin: 0 0 var(--sp-1);
496+
}
497+
498+
.model-hint,
499+
.speed-hint {
500+
margin: 0;
430501
font-size: 0.8125rem;
431-
line-height: 1.6;
432502
color: var(--muted);
433503
}
504+
505+
.model-hint:empty,
506+
.speed-hint:empty {
507+
display: none;
508+
}

0 commit comments

Comments
 (0)