Jakarta GPU Capacity: VRAM Sizing for Indonesian AI Teams

The question is small and the answer is not: which GPU do I rent to serve an Indonesian-language model from Jakarta? I opened the Sahabat AI 70B model card expecting a shopping list, and found one sentence that closes most of the options before the shopping starts. To load the model in FP16 or BF16 you need approximately 140 GB of VRAM, with two NVIDIA H100s or four L40s given as the recommended setups.

This post is the arithmetic behind that sentence, and what to do about it in a quarter when the large Jakarta AI capacity everyone is waiting for is contracted rather than rentable. It covers the four ways an Indonesian team can get a GPU today, how to size VRAM before renting anything, why fine-tuning and serving are different machines, and what quantisation genuinely buys and costs.

VRAM is the binding constraint, and it is arithmetic

Start with the finding: for inference you are not shopping for FLOPS, you are shopping for memory. A model that does not fit in GPU memory does not run slowly, it does not run at all. The first term is the easy one — parameters times bytes per parameter. At BF16 that is two bytes each, so a 70 billion parameter model is about 140 GB of weights before anything else is loaded, which is exactly the figure the Sahabat AI 70B card publishes.

// The easy term: parameters x bytes per parameter.
const BYTES_PER_PARAM = { bf16: 2, int8: 1, int4: 0.5 };

function weightsGB(paramsBillion, format) {
  return (paramsBillion * 1e9 * BYTES_PER_PARAM[format]) / 1e9;
}

weightsGB(70, "bf16"); // 140 — the figure the Sahabat AI 70B card publishes
weightsGB(70, "int8"); //  70 — one 80 GB card becomes conceivable
weightsGB(70, "int4"); //  35 — before format and scale overhead

// A model card gives you this line and stops. It is never the whole budget.

This is worth doing by hand because it is the only part of the decision you can settle before spending anything. Throughput, latency and cost per token all depend on traffic you may not have yet. Whether the weights fit on the card you are about to rent is a fact you can establish in ten seconds with a calculator, and it disqualifies most of the menu on its own.

The KV cache is the term people forget

The second term grows with your traffic rather than with your model. Every token already processed keeps a key and a value vector in memory for every layer, so the cache is two, times layers, times key-value heads, times head dimension, times bytes per element — per token, per sequence. The published config for the Sahabat AI 70B lists 80 layers, 8 key-value heads and a head dimension of 128, which at two bytes an element comes to 320 KiB of cache for a single token.

// Straight out of the model's published config.json:
const LAYERS = 80;    // num_hidden_layers
const KV_HEADS = 8;   // num_key_value_heads — GQA, not the 64 attention heads
const HEAD_DIM = 128; // head_dim
const BYTES = 2;      // bf16

// Key AND value, every layer, every token, every concurrent sequence.
const perToken = 2 * LAYERS * KV_HEADS * HEAD_DIM * BYTES;
// 327,680 bytes = 320 KiB per token

perToken * 8192;   //  2.7 GB — a modest 8k conversation
perToken * 131072; // 42.9 GB — ONE session using the advertised 128k window

// Concurrency multiplies this. Parameter count does not.

That number looks small and is not. The model advertises a 128k context window, and at 320 KiB per token one conversation that actually uses it needs about 43 GB of KV cache for one user — more than half of an 80 GB card. Grouped-query attention is already saving you here: if the cache were sized by the 64 attention heads rather than the 8 key-value heads, the same arithmetic would be eight times worse.

Two H100s hold the weights and not much else

Put the two terms together against the recommended hardware and the shape of the problem appears. NVIDIA lists GPU memory of 80 GB for the H100 SXM and 48 GB for the L40S, so the model card's two recommendations come to 160 GB and 192 GB of total memory for a model whose weights alone are about 140 GB. The final column below is what is actually left to serve users with.

Read the last column, not the first. The two-H100 setup that satisfies the model card leaves room for roughly sixty thousand tokens of cache across every concurrent request, which is not even one full-length conversation. The four-L40S setup, which reads like the budget option, is the one with headroom. And both rows are still optimistic, because they count only weights and cache and nothing the runtime needs.

The four ways to get a GPU this quarter

There are four real options and they differ less in price than in what they take away from you. A managed inference API removes the arithmetic above entirely and costs nothing while idle, at the price of serving whatever models the vendor lists and sending your prompts to their region. Renting GPU hours from an overseas neocloud gives you the widest menu of card types and hourly granularity, at the price of egress, a round trip out of Indonesia, and operating the server yourself.

Renting in-country is the option teams assume is a straight substitute and it is not: where it exists the menu of card types is narrower, so availability tends to decide your model rather than the other way round. Buying or colocating hardware is the only option that makes idle time free and every other cost yours — import, power, cooling, spares, and a card that depreciates whether or not anyone is chatting. I would not buy anything before I had a month of measured, saturated utilisation to point at.

Fine-tuning and serving are different machines

Conflating these is the most expensive mistake in this area, because a fine-tuning footprint is a multiple of a serving footprint rather than the same number. Serving holds weights plus KV cache. A full fine-tune holds weights, gradients the size of the trainable weights, optimiser state for every trainable parameter, and activations that scale with batch size and sequence length — all before a single user has been served.

The practical consequence is that the two jobs should be bought differently. Fine-tuning is bursty, finite and tolerant of a queue, which makes it the ideal thing to rent by the hour on whichever card is available this week, including outside Indonesia when the training data allows it. Serving is continuous, latency-sensitive and wants to sit near your users. Teams that size one cluster for both end up paying for training-shaped hardware twenty-four hours a day. Adapter methods such as LoRA change the fine-tuning term dramatically by training a small fraction of the parameters, and change the serving term not at all — you still load the full base model.

Quantisation is the highest-leverage lever you have

Bytes per parameter is the only term in the weights arithmetic you control, which is why quantisation moves the answer further than anything else here. Two bytes per parameter puts a 70B model at 140 GB; one byte puts it at 70 GB; four bits puts it near 35 GB before format and scale overhead. That is not a saving, it is a change of category — the difference between a two-card rental you have to reserve and a single 80 GB card you can take by the hour.

The honest cost is that quantised models take less memory at the cost of lower precision, as vLLM's documentation puts it, and how much quality you actually lose is specific to your model, your language and your task. There is a second trap worth knowing: quantising the weights does nothing to the KV cache, which is a separate allocation with its own dtype. If long contexts are what fills your card, the cache dtype is the switch you want, and vLLM exposes it as kv-cache-dtype independently of the weight format.

Contracted capacity is not capacity you can rent

The last thing to get right is time. In August 2026 CoreWeave announced three Indonesian data centres carrying 360 megawatts of contracted IT power in Greater Jakarta, expected online in 2028. That is a real commitment and a genuine change to what Indonesian teams will be able to buy — in 2028. Contracted power is not a card in a slot, and a startup planning this quarter around it is planning around hardware that does not exist yet.

So design for the hardware you can get today and make the model a configuration value rather than an architectural assumption. Put the model behind your own interface, keep the prompt and the eval portable, and treat the card underneath as something you will change twice before that capacity opens. The teams best placed in 2028 are the ones already serving real traffic on rented hours in 2026, because they will know exactly what to ask for.

What I would do this quarter

All three bands have to be budgeted before you can name a card, so the order of work is arithmetic first, hardware last. This is the sequence I would follow for an Indonesian product shipping now.

1. Compute the three terms before renting anything: weights at your chosen precision, cache at your real p95 context and concurrency, and a margin for runtime overhead.
2. Start on a managed API and log every request, so the eval and the traffic profile exist before the hardware decision does.
3. When you do self-host, rent by the hour and quantise to the smallest format that still passes your eval, rather than renting a second card to keep BF16.
4. Bound the server explicitly, so the failure mode under load is a rejected request rather than a crash.

# Bound every term explicitly. The defaults suit a benchmark, not your traffic.
vllm serve GoToCompany/Llama-Sahabat-AI-v2-70B-IT \
  --tensor-parallel-size 2 \
  --max-model-len 8192 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.92 \
  --kv-cache-dtype fp8

# --tensor-parallel-size 2  splits the 140 GB of weights across both H100s
# --max-model-len 8192      where the KV cache term is really bounded — not by the card
# --max-num-seqs 4          4 x 8192 x 320 KiB = about 10 GB, inside the 20 GB left over
# --gpu-memory-utilization  the remainder covers activations, CUDA graphs, fragmentation
# --kv-cache-dtype fp8      halves the cache; a separate decision from the weight format

# Wrong: leave max-model-len at the model default and meet 128k under load.
# Right: set it from the p95 prompt length in your own logs, then raise it deliberately.

That last step is one command, and it is the one most teams leave at defaults. Every flag below maps to a term in the arithmetic above, which is the point: the server should be configured from your own numbers, not from the model card's.

The rule worth carrying out of this is that memory gets sized before anything gets shopped for, and arithmetic disqualifies options faster than any vendor brochure. VRAM is three terms — parameters, cache and overhead — and all three are computable on paper in an afternoon. Jakarta will have far more capacity in 2028. What your team can serve this quarter is decided by the term you did not budget for.