Speech-to-Text Bahasa Indonesia in Production: Real Whisper WER

Here is a sentence an Indonesian customer-service agent says perhaps forty times a day: tolong follow up invoice yang outstanding itu ya, nanti saya forward ke finance. Thirteen words, five of them English, in Indonesian word order, spoken at conversational speed down an 8 kHz phone line. Every published Indonesian word error rate you can find was measured on audio that looks nothing like it.

This post is the numbers with their benchmarks attached. Whisper's own paper contains a per-language table almost nobody quotes, a 2024 Indonesian evaluation publishes a very different one for the same model, and a 2024 code-switching study publishes the figure that should worry you most. After that: where language detection goes wrong, the parameters that matter for Indonesian, how to chunk long call audio, and what a GPU costs against a per-minute meter.

The Indonesian WER everyone quotes is measured on audio you do not have

Start with the primary number. Appendix D of the Whisper paper contains Table 13, WER on FLEURS, broken down by language and model size. The Indonesian column reads: tiny 51.7, base 33.1, small 16.3, medium 10.2, large 8.5, large-v2 7.1. Those are OpenAI's own measurements on OpenAI's own models, and they are the origin of nearly every claim in circulation that Whisper does about 7 percent on Indonesian.

Now the second number. In October 2024 a team published an evaluation of multilingual models on Indonesian speech with deliberately mixed variabilities, assembled into an 80.54-hour corpus from parliamentary hearings, news, Common Voice, FLEURS, podcasts and talk shows. On that test set whisper-small scored 30.87 WER. The same model scores 16.3 on FLEURS. Nothing changed except the audio.

The split inside that paper is the part worth memorising. whisper-small managed 27.22 WER on read, formal, clean speech and 40.66 on spontaneous, informal, clean speech. That is thirteen points of degradation with no noise added at all, purely because the speaker stopped reading from a page. Their fine-tuned whisper-small then came in at 14.85 overall, under half the stock model's error. A word error rate quoted without its benchmark is a marketing number, and every figure below carries its benchmark with it.

The six options worth taking seriously

These do not sort cleanly into open and managed, and they do not sort by price either. The accuracy column always names its benchmark, and where a number is a vendor's own claim it says so.

Convert every price to cost per hour of audio before you compare anything. OpenAI publishes $0.006 a minute for whisper-1 and $0.003 for gpt-4o-mini-transcribe, Deepgram publishes $0.0043 a minute for Nova-3 pre-recorded on pay-as-you-go, and ElevenLabs publishes $0.22 per audio hour for Scribe v2 and $0.39 for its realtime tier. Per-minute and per-hour quotes sit a factor of sixty apart, and that is an easy mistake to make in a spreadsheet.

List prices as published by each vendor: OpenAI API pricing · ElevenLabs API pricing · Deepgram pricing

Code-switching is where Indonesian speech recognition actually breaks

Go back to that opening sentence. This is not borrowing. Follow up, invoice, outstanding and forward are English words with English phonology, dropped into Indonesian syntax by a speaker who is fluent in both and is not signalling a switch to anybody. Indonesian office speech is saturated with it, and so is clinic speech, field-service speech, and anything at all involving a system whose screens were specified in English.

Whisper's architecture makes one decision here that explains most of the damage. Its decoder emits a language token first, a single token drawn from a set of ninety-nine, predicted before any transcript text at all, and the paper is explicit that this happens once per thirty-second audio window. Everything after it is decoded conditioned on that choice. There is no way to express the idea that this window is Indonesian except for four words. The model has to pick a side, and then spell the other language the way the chosen side spells things.

The magnitude is documented. A 2024 study trained an ASR on 24 hours of English and 40 hours of Indonesian and measured character error rate three ways. Monolingual English: 2.43 percent. Monolingual Indonesian: 4.10 percent. Synthesised Indonesian-English code-switching: 37.57 percent. Natural, spontaneous, slightly noisy Indonesian-English code-switching: 91.76 percent. That is character error rate rather than word error rate, measured on a Speech-Transformer rather than on Whisper, so do not transplant the number. Transplant the shape: a model that reads Indonesian at 4 percent CER and code-switched Indonesian at 92 percent is not degrading, it is failing.

Their semi-supervised method, a machine speech chain trained on unlabelled code-switched audio, pulled the synthesised case down to 18.56 percent and the natural case only as far as 82.62 percent. The synthetic gain is large and the natural gain is small, which is itself the finding: clean code-switched audio is a tractable engineering problem, and noisy spontaneous code-switched audio is still open research.

The language parameter decides more than the model does

The most consequential line in a Whisper call is the one people leave out. Read whisper/transcribe.py: if you do not pass language, it runs detect_language on up to the first thirty seconds, takes the argmax, writes the winner into decode_options and reuses it for the whole file. One guess, taken from the opening, applied to everything after it. An Indonesian call that opens with an English greeting and an English company name is a call where the detector has a real chance of choosing English and then decoding forty minutes of Indonesian under an English prior.

# openai-whisper, transcribing one Indonesian customer-service call.

# WRONG - no language argument.
# transcribe() calls detect_language() on up to the FIRST 30 SECONDS only, takes
# the argmax, writes it into decode_options and reuses it for the entire file.
# A call opening "Hello, selamat pagi, dengan Rina di sini" can lock <|en|> and
# then decode forty minutes of Indonesian under an English prior.
result = model.transcribe("call-2026-09-12.wav")

# RIGHT - pin the language, then fix the two defaults that hurt long call audio.
result = model.transcribe(
    "call-2026-09-12.wav",

    language="id",       # emits <|id|> in every 30-second window; no detection runs
    task="transcribe",   # "translate" is X->English. It will hand you fluent
                         # English text and look exactly like a successful run.

    condition_on_previous_text=False,
    # Defaults to True: each window's output becomes the next window's prompt.
    # Good for consistency, ruinous for call audio - one bad window starts a
    # repetition loop that every later window inherits. The docstring says so.

    compression_ratio_threshold=2.4,  # gzip ratio of the decoded text; above this
                                      # the segment counts as failed and is retried
                                      # further up the temperature ladder
    logprob_threshold=-1.0,           # below this, same treatment
    no_speech_threshold=0.6,          # raise it on noisy 8 kHz lines, or hold
                                      # music comes back as confident words

    initial_prompt=(
        "Transkrip layanan pelanggan. "
        "Istilah yang muncul: invoice, outstanding, follow up, "
        "PPN, resi, COD, refund, nomor order."
    ),
    # The prompt does NOT change the language token. It raises the probability of
    # these exact English spellings so the Indonesian prior stops respelling them.

    word_timestamps=True,
)

Pinning the language to id fixes that and creates a smaller problem in its place: every English word in the audio is now decoded under an Indonesian prior. For Indonesian business audio that is the correct trade, because Indonesian is the matrix language and the embedded English is a minority of tokens. It is still a trade rather than a fix, and initial_prompt is the lever that partly compensates. Priming the decoder with the English terms you expect, spelled in English, raises their probability without touching the language token at all.

from openai import OpenAI

client = OpenAI()

# Same trap, wider blast radius: detection runs PER REQUEST. A forty-minute call
# split into twelve uploads gets twelve independent language guesses, so one
# English-heavy chunk flips mid-call while every other chunk stays Indonesian.
with open("call-2026-09-12.part01.mp3", "rb") as audio:
    r = client.audio.transcriptions.create(
        model="whisper-1",
        file=audio,            # 25 MB hard limit per request - hence the chunking
        language="id",         # ISO 639-1. Omit it and you are back to guessing.
        prompt="invoice, outstanding, follow up, PPN, resi, COD, refund",
        # whisper-1 truncates this at 224 tokens. Spend them on the code-switched
        # nouns you actually expect, not on a polite instruction sentence.
        response_format="verbose_json",
        timestamp_granularities=["word"],   # verbose_json + whisper-1 only
        temperature=0,
    )

seg = r.segments[0]
print(seg.avg_logprob, seg.no_speech_prob)
# Log both. Whisper's own decoder treats avg_logprob below -1.0 as a failed
# segment, so a run of segments hugging that line is the cheapest early warning
# that the problem is the audio and not the model.

The managed APIs carry the same trap with a wider blast radius, because detection runs per request and a long recording is many requests. Pin the language on every chunk, or accept that chunk seven can disagree with chunk six about what language the call is in.

Chunking, silence, and the twenty-five megabyte wall

OpenAI's transcription endpoint caps uploads at 25 MB, and its guide tells you to split larger files while avoiding splits in the middle of a sentence. The second half of that instruction is the half that matters and the half people skip. Cutting a two-hour recording every ten minutes on a clock will land some cuts inside words, and Whisper does not hand back half a word on either side of the seam. It hands back a confident whole word that was never spoken.

from silero_vad import load_silero_vad, read_audio, get_speech_timestamps

SR = 8000         # the rate the call recording actually IS. Do not upsample here:
                  # Silero handles 8 kHz natively, and one resample later in the
                  # pipeline is already one more than zero.
MAX_CHUNK_S = 600 # ten minutes. Well under the 25 MB upload cap for mono audio,
                  # and short enough that a failed chunk is cheap to retry.

model = load_silero_vad()
wav = read_audio("call-2026-09-12.wav", sampling_rate=SR)
speech = get_speech_timestamps(wav, model, sampling_rate=SR, return_seconds=True)


def pack(regions, max_s):
    """Group speech regions into chunks whose boundaries all land in silence."""
    chunks, start, end = [], None, None
    for r in regions:
        if start is None:
            start, end = r["start"], r["end"]
        elif r["end"] - start > max_s:
            chunks.append((start, end))         # close at the END of the previous
            start, end = r["start"], r["end"]   # region, i.e. inside the gap
        else:
            end = r["end"]
    if start is not None:
        chunks.append((start, end))
    return chunks


for i, (a, b) in enumerate(pack(speech, MAX_CHUNK_S)):
    # -c copy cuts the ORIGINAL file, so re-encode artefacts never stack up.
    print(f"ffmpeg -i call.wav -ss {a:.2f} -to {b:.2f} -c copy part{i:03d}.wav")

Cut on silence instead. Run a voice activity detector over the file, take the speech regions, pack them into chunks up to your size or duration budget, and place every boundary inside a gap. It costs almost nothing and it removes an entire class of error that is otherwise invisible until somebody reads the transcript closely.

Call-centre audio is eight kilohertz, and that is a ceiling

Whisper resamples every input to 16,000 Hz before computing its mel spectrogram. That is stated in the paper's architecture section and it is not a setting. Indonesian call-centre recordings arrive at 8 kHz because that is what the telephony path carries, and resampling 8 kHz up to 16 kHz restores nothing, because there is no signal above 4 kHz in the file to restore. The model is running on half the bandwidth its training distribution had.

I have not seen an Indonesian-specific measurement of what that costs, and I am not going to invent one. What I will say is that it makes every published figure in this post inapplicable to telephone audio without re-measurement, because FLEURS and Common Voice are both wideband. Three things are worth doing before you accept the loss:

• Ask for the recording from before the mixdown. Most call platforms can store the agent and the customer on separate channels. Two mono files transcribed separately beats one mixed file plus speaker diarisation, and it is exact rather than probabilistic.
• If all you have is mono, treat diarisation as the fallback and not the default. WhisperX wraps Whisper with pyannote diarisation and forced alignment, so budget for the extra GPU time and for accepting the gated model terms on Hugging Face before any of it will run.
• Keep one sample rate end to end. Silero VAD at 8 kHz on 8 kHz audio is correct, and upsampling everything to 16 kHz first so that the pipeline looks uniform adds a resample and buys nothing.

The general rule is that a pipeline should touch the audio as few times as it can, and every conversion in it should be one you can name a reason for.

What self-hosting costs against a per-minute meter

The compute is cheap and the arithmetic is not intuitive, so here it is in full. The faster-whisper README benchmarks large-v2 at fp16 with batch size 8, transcribing 13 minutes of audio in 17 seconds on an RTX 3070 Ti, which works out at about 46 times real time. RunPod lists an RTX 4090 at $0.34 an hour on its community tier. Those two published numbers are the only input the calculation needs.

# Break-even, derived from two published numbers and nothing else.
#
#   faster-whisper README - large-v2, fp16, batch_size 8, RTX 3070 Ti 8GB:
#     13 minutes of audio in 17 seconds  ->  13*60/17 = 45.9x real time
#   RunPod list price - RTX 4090, community tier:
#     $0.34 per GPU-hour

GPU_HOUR = 0.34
REALTIME_FACTOR = 45.9
MONTH_HOURS = 720          # a GPU rented by the month is billed for all of them,
                           # busy or idle. That is the entire argument.

gpu_month = GPU_HOUR * MONTH_HOURS            # $244.80

# Managed list prices, every one converted to dollars per HOUR OF AUDIO first.
managed = {
    "OpenAI whisper-1": 0.006 * 60,           # $0.360/h
    "Deepgram Nova-3": 0.0043 * 60,           # $0.258/h  pre-recorded, pay-as-you-go
    "ElevenLabs Scribe v2": 0.22,             # $0.220/h
}

for name, rate in managed.items():
    audio_hours = gpu_month / rate
    gpu_hours = audio_hours / REALTIME_FACTOR
    print(f"{name:22} {audio_hours:6.0f} audio-hours to break even, "
          f"{gpu_hours:5.1f} GPU-hours of real work, "
          f"{gpu_hours / MONTH_HOURS:.1%} duty cycle")

# OpenAI whisper-1          680 audio-hours to break even,  14.8 GPU-hours of real work, 2.1% duty cycle
# Deepgram Nova-3           949 audio-hours to break even,  20.7 GPU-hours of real work, 2.9% duty cycle
# ElevenLabs Scribe v2     1113 audio-hours to break even,  24.2 GPU-hours of real work, 3.4% duty cycle

The two inputs, both published: faster-whisper benchmarks · RunPod GPU pricing

So a GPU left running all month costs roughly what 680 hours of OpenAI transcription costs, or 949 hours of Deepgram, or 1,113 hours of ElevenLabs Scribe. Below about 23 hours of audio a day the managed API is simply cheaper, and you have written no code to get there. Above it the GPU wins on paper, but look at the duty cycle: 680 audio hours at 46 times real time is under 15 GPU-hours, so the machine you rented by the month is idle for 98 percent of it.

Which makes the honest version of the argument this. Rent the GPU by the second rather than by the month and the compute cost collapses to a few dollars, at which point what you are really paying for is the orchestration, the cold starts, the retry logic and the person who gets paged at two in the morning. Price is not the reason to self-host. Audio that is not allowed to leave your own infrastructure is a reason. Wanting to fine-tune on your own Indonesian data, which that 14.85 result says is worth doing, is a reason. Saving two hundred dollars a month is not.

What the published numbers will not tell you

Every WER in this post, including the ones I trust most, was computed after text normalisation, and the Whisper paper is open about having developed its own normaliser and about it generally producing lower WERs than the alternative it was compared against. Normalisation is what decides whether Rp 1.500.000 and satu juta lima ratus ribu rupiah are the same answer. For Indonesian, with its numerals, its titles and its inconsistent spacing around prefixes, that single choice moves the score more than most model choices do.

So treat published WER as a floor. Benchmark audio is read aloud by a cooperative speaker into a decent microphone at full bandwidth in one language. Your audio is a stressed person on a phone in a room with a fan, switching to English for every noun that came out of an ERP screen, in a regional accent the training data barely saw. The only number that predicts your production experience is one you measure yourself.

1. Pull thirty to sixty minutes of real recordings, sampled across your actual conditions rather than across the clean ones.
2. Transcribe them by hand once, and write down the normalisation rules you applied while you were doing it.
3. Score every candidate model against that reference with the same normaliser, and record which benchmark each vendor number came from so the gap stays visible.
4. Re-run it whenever you change chunking, sample rate or the language parameter. All three move the score, and only one of the three is the model.

That reference set is an afternoon of work and it will be the most useful asset in the project. It is also the only thing that will tell you whether code-switching is 5 percent of your tokens or 30 percent, and that number decides whether any of this is viable at all.

Treat every published Indonesian word error rate as a statement about a benchmark rather than about your audio. Pin the language parameter, cut on silence rather than on a clock, keep the sample rate you were given, and build a thirty-minute in-house reference set before you choose anything. Those four decisions will move your transcript quality further than the choice of model will.