Telling not-allowed From no-speech in SpeechRecognition

The short version

Unless SpeechRecognitionErrorEvent.error is split into “a real failure” and “a benign code”, the mic button keeps looking like it does nothing. not-allowed (permission denied), audio-capture (no microphone found) and unsupported (API not available) are failures the user should be told about, while no-speech (a silence timeout) and aborted (a normal interruption) happen routinely in ordinary use — hint on those and the UI becomes noisy with false alarms. We collected that decision into a pure function, sttErrorHint, and pinned it with unit and render tests to cover what a real microphone cannot in CI.

What it looks like

On the conversational AI assistant screen (EditorChat), a mic button (🎤) sits next to the text field; pressing it makes the useSpeechToText hook start the Web Speech API’s SpeechRecognition.

{stt.supported ? (
  <button
    type="button"
    style={stt.listening ? micActiveStyle : micStyle}
    onClick={() => {
      if (streaming) return;
      if (stt.listening) stt.stop();
      else stt.start();
    }}
    aria-label={stt.listening ? "音声入力を止める" : "音声入力"}
    aria-pressed={stt.listening}
    title="音声入力"
  >
    🎤
  </button>
) : null}

Before the fix, this button had no code referencing stt.error at all. Press it while the browser’s permission is denied, while no microphone is physically present, or in a browser with no Web Speech API, and recognition stops with an error on SpeechRecognitionErrorEvent. The screen displayed that error nowhere, so from the user’s point of view pressing the mic button did nothing at all. Review on PR #876 raised this, and it was handled as non-blocking (does not block the merge, but should be fixed).

Why

The previous implementation simply never displayed stt.error anywhere; there was no branching by error kind to begin with. The hook piled codes into the same error state on every failure and left the display to the caller.

const Ctor = getRecognitionCtor();
if (!Ctor) {
  setError("unsupported");
  return;
}

recognition.onerror = (event) => {
  setError(event.error);
};

Naively implementing “always warn when error is set” creates a different problem, because SpeechRecognition sets error in situations that are not failures.

• no-speech … nothing was said for a while (a timeout). You just speak again; not a failure
• aborted … recognition was interrupted normally (user action, restart, navigation, etc.)

Both occur routinely if you use voice input at all. Warning on them makes something appear every time the mic is used, which buries the hint for the states where the user genuinely is stuck (permission denied, no microphone). “Ignore every error” and “show every error” fail in the same way: the user cannot tell when the microphone is actually broken.

Fixing it

We collected the mapping from an error code to the hint text (or null for no hint) into a pure function, sttErrorHint, separated from the UI.

// apps/web/lib/teacher-input/stt-error-hint.ts
const BENIGN_STT_ERROR_CODES: ReadonlySet<string> = new Set(["no-speech", "aborted"]);

export function sttErrorHint(error: string | null): string | null {
  if (!error || BENIGN_STT_ERROR_CODES.has(error)) {
    // Nothing happened (null / empty) or a benign code — not a failure, so no hint.
    return null;
  }
  switch (error) {
    case "not-allowed":
    case "service-not-allowed":
      return "マイクを使えませんでした。ブラウザの設定でマイクの使用を許可してください。";
    case "audio-capture":
      return "マイクが見つかりませんでした。接続を確認してもう一度お試しください。";
    case "unsupported":
      return "このブラウザは音声入力に対応していません。キーボードで入力してください。";
    default:
      // network and other unexpected codes are still "pressed it and nothing worked" failures,
      // so give a generic hint.
      return "音声入力を開始できませんでした。もう一度お試しください。";
  }
}

unsupported is not a code the Web Speech API returns; it is a synthetic code the useSpeechToText hook sets on detecting an unsupported browser. Inside sttErrorHint it is treated on equal terms with the other failure codes and produces a hint.

The caller, EditorChat, is kept as a thin layer that renders the return value beneath the mic button with role="status".

// apps/web/app/app/editor/_components/EditorChat.tsx
// Pick up only the real voice-input failures (permission denied, no mic, unsupported, etc.)
// and show a hint. Benign codes (no-speech / aborted) and "nothing happened" are null
// (avoiding false alarms). The split lives in a pure function (tested).
const micHint = sttErrorHint(stt.error);

{/* Only when voice input actually failed, show a short hint under the mic (announced via role=status). */}
{micHint ? (
  <p role="status" style={micHintStyle}>
    {micHint}
  </p>
) : null}

role="status" is used so a screen reader announces the change without the hint having to be seen. The button’s own appearance is unchanged; a short line simply appears beneath it on failure.

Preventing a repeat

Real microphone behaviour — how the permission dialog is answered, an actually disconnected mic — cannot be reproduced in CI. So this fix extracted only the decision “which error codes get a hint and which do not” into the pure function sttErrorHint and pinned it in unit tests that do not depend on hardware.

// apps/web/__tests__/teacher-input/stt-error-hint.test.ts
it("benign codes (no-speech / aborted) produce no hint (avoids false alarms)", () => {
  expect(sttErrorHint("no-speech")).toBeNull();
  expect(sttErrorHint("aborted")).toBeNull();
});

it("permission denial (not-allowed / service-not-allowed) hints at granting permission", () => {
  for (const code of ["not-allowed", "service-not-allowed"]) {
    const hint = sttErrorHint(code);
    expect(hint).not.toBeNull();
    expect(hint).toContain("許可");
  }
});

On top of that, EditorChat gained a render test that mocks useSpeechToText to swap stt.error, pinning only the display-side wiring: “no role="status" element appears for a benign code” and “one appears for a real failure”. Because the correctness of the logic and its reflection on screen are guaranteed by separate tests, a future change to how error codes are handled will fail a test if it tips the wrong way — adding false alarms, or hiding a real failure.