RAG Bahasa Indonesia: Postgres Full-Text Search Misses Docs

A keyword query that should have matched returned nothing. The phrase was pembayaran yang belum diterima, payment not yet received, and the stored document said pembayaran belum diterima dari pelanggan. The same words, one connector dropped. Postgres full-text search, the indonesian configuration, no match. On 18 September 2026 I opened psql against PostgreSQL 16.11 and spent the afternoon taking that configuration apart.

Everything below was measured on that instance and every line of it is reproducible with psql. The finding is not that Postgres lacks Indonesian support, because it has it, and that is precisely the problem. The configuration exists, it looks correct in the catalogue, and it quietly indexes every stop word and mis-stems a large share of the verbs you will actually search for.

Postgres 16 does ship an Indonesian configuration

This is where most write-ups stop, and they stop at the wrong place. PostgreSQL 16.11 ships 29 text search configurations and indonesian is one of them, alongside arabic, hindi, nepali and tamil. to_tsvector with the indonesian configuration works, a GIN index over it builds, EXPLAIN shows the index being used, and nothing anywhere warns you. So you tick Indonesian support off the list and move on, which is what I did.

-- PostgreSQL 16.11. The trap is not a missing config, it is a missing
-- parameter on the dictionary INSIDE a config that already exists.
SELECT c.cfgname, d.dictname, d.dictinitoption
FROM   pg_ts_config c
JOIN   pg_ts_config_map m ON m.mapcfg = c.oid
JOIN   pg_ts_dict d       ON d.oid    = m.mapdict
WHERE  c.cfgname IN ('english', 'indonesian')
GROUP  BY 1, 2, 3 ORDER BY 1, 2;

--   cfgname   |    dictname     |             dictinitoption
-- ------------+-----------------+---------------------------------------------
--  english    | english_stem    | language = 'english', stopwords = 'english'
--  indonesian | indonesian_stem | language = 'indonesian'
--
-- No stopwords parameter. No warning, no notice, no error at CREATE INDEX time.

The difference from English hides in one column of pg_ts_dict that nobody reads. english_stem is created with a language parameter and a stopwords parameter. indonesian_stem is created with a language parameter and nothing else. There is a matching absence on disk: no indonesian.stop file is installed anywhere in the tsearch_data directory, so there is nothing for the missing parameter to point at even if you added it.

Every yang and dan is indexed, because no stop list ships

A thirteen-word Indonesian sentence produced thirteen lexemes. The nineteen-word English control produced eight. Nothing at all was dropped from the Indonesian side, not yang, not dan, not di, not dari, not belum. Run the two side by side and the asymmetry is impossible to miss, but only if you think to run them, and nothing in the tooling suggests you should.

-- 13 Indonesian words in.
SELECT to_tsvector('indonesian',
  'Pelanggan yang sudah melakukan pembayaran di aplikasi dan belum menerima invoice dari sistem');

-- 'aplikasi':7 'bayar':5 'belum':9 'dan':8 'dari':12 'di':6 'erima':10
-- 'invoice':11 'laku':4 'langgan':1 'sistem':13 'sudah':3 'yang':2
--                                        13 lexemes out. Nothing dropped.

-- 19 English words in, as a control.
SELECT to_tsvector('english',
  'Customers who have made a payment in the app and have not yet received an invoice from the system');

-- 'app':9 'custom':1 'invoic':16 'made':4 'payment':6 'receiv':14 'system':19 'yet':13
--                                        8 lexemes out. 11 words dropped.

-- The same asymmetry on bare function words:
SELECT to_tsvector('indonesian', 'yang dan di dari sudah belum');
--   'belum':6 'dan':2 'dari':4 'di':3 'sudah':5 'yang':1   -- all six kept
SELECT to_tsvector('english',    'the and in from already not');
--   'alreadi':5                                            -- five of six dropped

This is a missing file rather than a tuning preference. Postgres installs fifteen stop word lists in tsearch_data: danish, dutch, english, finnish, french, german, hungarian, italian, nepali, norwegian, portuguese, russian, spanish, swedish and turkish. english.stop holds 127 words. Indonesian gets none, so yang, which turns up in a large share of Indonesian sentences, ends up with a posting list roughly the size of your table, and every ranking function you call has to weigh it like any other term.

The real damage is in the query, not in the index

Keeping stop words in the index costs storage. Keeping them in the query costs recall, and recall is what reaches you as a bug report. plainto_tsquery joins the terms it extracts with AND, so every function word your user happened to type becomes a term the document is required to contain.

-- plainto_tsquery joins its terms with AND. Every stop word the user typed
-- becomes a term the document is REQUIRED to contain.
SELECT plainto_tsquery('indonesian', 'pembayaran yang belum diterima');
--   'bayar' & 'yang' & 'belum' & 'terima'      -- 4 required terms, 2 meaningless

SELECT plainto_tsquery('english', 'payment that has not been received');
--   'payment' & 'receiv'                       -- 2 required terms

-- And here is the bug report you will actually receive.
SELECT doc, to_tsvector('indonesian', doc)
            @@ plainto_tsquery('indonesian', 'pembayaran yang belum diterima') AS hit
FROM unnest(ARRAY[
  'Pembayaran belum diterima dari pelanggan',        -- f  no 'yang', no match
  'Pembayaran yang belum diterima dari pelanggan',   -- t
  'Invoice ini belum dibayar oleh pelanggan'         -- f
]) AS doc;

-- Dropping 'yang' is a style choice in Indonesian, not a change of meaning.
-- The index treats the two sentences as different documents.

Four required terms, two of which carry no meaning. The English control produced two. And a document that writes the same sentence without yang, which in Indonesian is a matter of register rather than meaning, cannot match at all. I measured that exact pair: the version with yang matches, the version without it does not, and no part of the system considers this remarkable.

The stemmer strips me- without restoring the consonant it ate

Indonesian builds verbs with the me- prefix, and the prefix assimilates to the first sound of the root, absorbing it. me- plus tulis gives menulis, the t swallowed by the n. me- plus kirim gives mengirim, the k absorbed into the ng. me- plus masak gives memasak. To a reader the root consonant has not disappeared, it is encoded in the nasal, and every Indonesian speaker unpacks it without thinking.

Snowball's Indonesian algorithm, which is what Postgres uses, implements the Porter-style stemmer for Bahasa Indonesia described in Fadillah Z Tala's 2003 thesis. It removes the prefix, including the variants men, meng, meny and mem, but it does not put the swallowed consonant back. So menerima loses men and becomes erima, while terima on its own is misread as a ter- prefix plus a root ima. One verb family splinters into three separate lexemes: terima gives ima, diterima gives terima, and menerima and penerimaan both give erima.

Look at the memasak row again. The stem is not merely wrong, it is pasak, which is a real Indonesian word meaning a peg or a dowel. I checked both directions: a query for pasak matches a document about cooking rice, and a query for memasak matches a document about hammering wooden pegs. That is not a near miss, it is a silent collision between two unrelated topics, and nothing in your logs will ever mention it.

It is not broken, it is unevenly correct, which is worse

Three of the seven verbs I tested stem correctly. menyimpan, memproses and mencatat all reduce to their roots. The bayar family is flawless: bayar, membayar, pembayaran, dibayar, membayarkan and pembayar all produce the single lexeme bayar, which is textbook behaviour and exactly what you hope a stemmer does.

-- Search the root, see whether the inflected document comes back.
SELECT q, doc, to_tsvector('indonesian', doc) @@ plainto_tsquery('indonesian', q) AS hit
FROM (VALUES
  ('terima', 'Kami sudah menerima barang dari vendor'),   -- f
  ('kirim',  'Barang akan dikirim besok'),                -- t   di- is stripped fine
  ('kirim',  'Kami mengirim barang besok'),               -- f   meng- is not
  ('masak',  'Juru masak sedang memasak nasi'),           -- t   matched the bare root
  ('bayar',  'Pelanggan sudah membayar tagihan')          -- t
) AS v(q, doc);
-- 2 misses out of 5. Same root, two prefixes, one of them works.

-- Now the family that makes you stop testing:
SELECT w, to_tsvector('indonesian', w)
FROM unnest(ARRAY['bayar','membayar','pembayaran','dibayar','membayarkan','pembayar']) AS w;
-- every one of the six gives 'bayar'. Perfect. So nobody checks the next verb.

That is what makes this expensive. Spot-check with bayar, conclude the stemmer works, ship. End to end, searching a root missed the inflected document two times out of five. kirim matched a document containing dikirim and failed on one containing mengirim, the same root with two prefixes, only one of which survives. There is no error, no log line and no metric that moves. The only signal is a user telling you the search is bad, months later.

Fix the stop words first, because Snowball publishes the list

Snowball's own Indonesian page publishes a stop word list. Postgres simply never installs it. The published file is 91 lines and flattens to 93 distinct words, and putting it into tsearch_data with a dictionary that references it took me under twenty minutes. My test sentence went from thirteen lexemes to eight, the four-term query collapsed to two terms, and the document that had been missing started matching.

# Snowball publishes the list; Postgres simply never installs it.
# The published file is annotated -- word, tab, a pipe, an English gloss --
# and a few lines hold two words. Postgres wants one bare word per line and
# does no other processing, so strip the comments and split the pairs first.
curl -sL https://snowballstem.org/algorithms/indonesian/stop.txt |
  awk -F'|' '{ print $1 }' | tr -s ' \t' '\n' | sed '/^$/d' | sort -u \
  > "$(pg_config --sharedir)/tsearch_data/indonesian.stop"     # 93 words

-- Same Snowball stemmer, now with the stop list the shipped config omits.
CREATE TEXT SEARCH DICTIONARY indonesian_stop_stem (
  TEMPLATE  = snowball,
  Language  = indonesian,
  StopWords = indonesian        -- resolves to tsearch_data/indonesian.stop
);

CREATE TEXT SEARCH CONFIGURATION id_search (COPY = indonesian);
ALTER TEXT SEARCH CONFIGURATION id_search
  ALTER MAPPING FOR asciiword, asciihword, hword_asciipart,
                    word, hword, hword_part
  WITH indonesian_stop_stem;

-- Same sentence as before: 13 lexemes becomes 8.
-- And the query that missed now matches:
SELECT to_tsvector('id_search', 'Pembayaran belum diterima dari pelanggan')
       @@ plainto_tsquery('id_search', 'pembayaran yang belum diterima');   -- t

-- because the query itself shrank from four required terms to two:
SELECT plainto_tsquery('id_search', 'pembayaran yang belum diterima');
--   'bayar' & 'terima'

Two traps before you copy the file. It is annotated, each line being a word, a tab, a pipe and an English gloss, and a few lines hold two words. Postgres wants one bare word per line and does no other processing, so strip the comments and split the pairs first. Second, the published list is incomplete for business text: it contains telah but not sudah, and sudah appears constantly in Indonesian operational writing. Extend it with your own words and keep the file in version control.

What the stop list does not fix, and three fallbacks

The stemmer is untouched by any of that. menerima still produces erima after the fix, and it always will, because the stop list and the stemmer are independent parts of the dictionary. Three options remain and none of them is free.

1. Drop to the simple configuration and normalise the text yourself: no stemming at all, plus your own prefix rules. Resist the obvious regex. Stripping a leading me, mem, men, meng or meny with a pattern turns meja, which means table, into ja, and still turns mengirim into irim. A pattern cannot tell a verb from a noun, and the shipped stemmer at least gets meja right.
2. Add pg_trgm as a fuzzy net. Measured, the similarity between terima and menerima is 0.3333, which clears the 0.3 default that show_limit reports, so the percent operator finds it. Between terima and penerimaan it is 0.2 and does not. Trigrams recover some Indonesian morphology by accident rather than by design, and they suit short typed fields rather than document bodies.
3. Accept that the lexical half is structurally weak for Indonesian and give the dense half more weight when you fuse the two result sets. This is the honest answer rather than the clever one, and it is where the rest of this post goes.

-- Option 1, the tempting regex. Do not ship this.
SELECT w, regexp_replace(w, '^(me|mem|men|meng|meny)', '') AS naive
FROM unnest(ARRAY['menerima','mengirim','memasak','menulis','meja']) AS w;
--   menerima -> erima     still wrong
--   mengirim -> irim      still wrong
--   meja     -> ja        "table" is now a fragment; the rule has no idea
--                         which words are even verbs

-- Option 2, trigrams as a fuzzy net. Recovers some morphology by accident.
CREATE EXTENSION IF NOT EXISTS pg_trgm;
SELECT w, similarity('terima', w), 'terima' % w AS passes_default_threshold
FROM unnest(ARRAY['terima','diterima','menerima','penerimaan']) AS w;
--   terima     1.0          t
--   diterima   0.4545       t
--   menerima   0.3333       t     -- clears the 0.3 default from show_limit()
--   penerimaan 0.2          f     -- does not
-- Good enough for a short field. Not a document search.

In a production system I would run all three, in that order of effort: the stop list on day one, a trigram index on the short fields people actually type into, and a fusion weight that does not pretend ts_rank over Indonesian text is as trustworthy as ts_rank over English text. The third is a judgement call, and writing it down as a number in your config is better than leaving it implicit.

Hybrid retrieval, and what the embedding numbers actually say

SEA-BED is the Southeast Asian embedding benchmark, 169 datasets across 10 languages and 9 task types. In Table 2 of the SEA-Embedding paper the Indonesian column reads: SEA-Embedding-E5-Large-600M 0.808, Cohere-embed-multilingual-v3.0 0.797, multilingual-e5-large-instruct 0.795, bge-m3 dense 0.781, Qwen3-Embedding-0.6B 0.756. Those are language-average scores across all nine task types, not retrieval-only scores, and quoting them as retrieval numbers would be wrong.

Read the spread rather than the ranking. On Indonesian the gap between the strongest and the weakest of those models is about five points. The same models' Lao column in the same table runs from 0.841 down to 0.298. Indonesian is one of the best-served languages in every serious multilingual encoder, which tells you where your recall is really going: not into the embedding choice, but into the lexical half you configured in ten seconds. MMTEB, covering over 500 tasks in more than 250 languages, points the same way by naming multilingual-e5-large-instruct at 560 million parameters as its best publicly available model, ahead of models with billions.

One thing I did not benchmark and will not assert: chunk size. Indonesian affixation makes words longer in characters than their English equivalents, so a chunker splitting on a character budget tuned for English lands in different places relative to sentence boundaries. Measure your own chunk lengths with your own tokenizer before reusing an English default, and treat any chunk-size number quoted at you without a named tokenizer as decoration.

Enabling the indonesian text search configuration is not the same thing as supporting Indonesian search, and the difference is invisible from the catalogue. Install a stop word list before you index anything. Test every affix family your domain actually uses rather than the one that happens to work. Treat Postgres full-text search as the weaker half of an Indonesian hybrid retriever rather than the half you tune. If you only do one of those, do the stop list: it is twenty minutes and it moves recall the same day.