# LangChain vs Building Your Own LLM Orchestration: When to Use Each

> An honest assessment of LangChain in 2025 — when it saves time, when it gets in the way, and how to decide if you need it at all.

URL: https://www.matthewswong.com/en/blog/langchain-vs-custom-llm/

- Author: Matthews Wong (https://www.matthewswong.com/en/about/)
- Published: 2025-06-10
- Updated: 2025-06-10
- Category: AI

I've built LLM applications both ways: using LangChain as the orchestration layer, and building custom orchestration from scratch. My AI Gymbro fitness app started with LangChain and was rewritten as custom orchestration six weeks in.

| Dimension | LangChain / LangGraph | Custom Orchestration |
| --- | --- | --- |
| Best for | Multi-provider apps, fast RAG prototypes | Simple workflows, 1-3 LLM calls per request |
| Setup speed | RAG pipeline in about 50 lines of Python | Same pipeline takes about 200 lines, more upfront work |
| Debugging | Stack traces run through 5 layers of internals | Errors surface directly from your own code |
| Provider switching | Swap OpenAI, Anthropic, Bedrock with no code changes | Each provider integration is hand-written |
| Complex stateful agents | LangGraph handles branching, multi-turn state well | Gets harder to manage past a few states |
| Token and latency control | Abstraction layer adds overhead | About 30% fewer tokens, about 25% lower latency in practice |
| Version stability | Breaking changes across major versions, pin carefully | No framework upgrades to manage |

## What LangChain Actually Is (and Isn't)

LangChain is a Python/JavaScript framework that provides abstractions for common LLM patterns: chains (sequences of LLM calls), agents (LLM + tools + loop), retrieval (vector store integration), and memory (conversation history management). The value proposition: a unified interface across multiple LLM providers, built-in implementations of common patterns, and a growing ecosystem of integrations.

### The Abstraction Tax

Every abstraction layer adds debugging distance between your code and the underlying API. When a LangChain chain fails, you get a stack trace through 5 layers of LangChain internals before you see the actual API error. I spent 3 hours debugging a SequentialChain that was silently dropping outputs.

### Where LangChain Genuinely Saves Time

LangChain is valuable for three use cases: rapid prototyping (get a RAG pipeline running in 50 lines of Python vs 200+ lines custom), multi-provider support (switch between OpenAI, Anthropic, and Bedrock without code changes), and complex agent workflows via LangGraph.

> If you're using LangChain, integrate LangSmith from day one. It captures full trace data — every LLM call, every chain step, every token count — and makes debugging dramatically easier. The free tier is generous enough for development and small-scale production.

## When to Build Custom Orchestration

Build custom orchestration when: your workflow is simple (1-3 LLM calls per user request); you need precise control over token usage and caching; you need streaming output with specific UI behavior; or your team is more comfortable debugging JavaScript/Python than framework internals.

### LangGraph for Complex Workflows

LangGraph is worth separating from LangChain proper. The graph-based workflow model (nodes are actions, edges are state transitions) is genuinely useful for complex multi-step agent workflows that have branching paths, conditional execution, and state that needs to persist across multiple turns.

> **LangChain Versioning Hell**

> LangChain has had numerous breaking changes between major versions. If you build production systems on LangChain, pin your versions strictly in requirements.txt and budget time for version migration work every 6-12 months. I've seen teams lose entire weekends to LangChain upgrade problems.

## The Decision Framework

Use LangChain/LangGraph if: you're building a RAG application and want fast iteration on retrieval strategies; you need multi-provider support; you're building complex stateful agent workflows; or your team already knows LangChain. Skip LangChain if: your workflow has fewer than 3 LLM calls; you need precise token control; or debuggability is critical.

## What I'd Do Now If Starting Fresh

For a new LLM project in 2025, I'd start with direct API calls using Anthropic or OpenAI's official SDK (both are excellent), add LangSmith for observability (it works without LangChain), and introduce LangGraph only if the workflow complexity genuinely warrants it.

## The Ecosystem Factor

One genuine advantage of LangChain in 2025 is the ecosystem: 600+ integrations, a large community, and LangSmith for observability. If you need to integrate with specific vector databases, LangChain's pre-built connectors save real time. The question is whether the ecosystem value outweighs the abstraction cost for your specific use case.

## Sources & Further Reading

- LangChain — Documentation — https://python.langchain.com/docs/introduction/
- LangGraph — Documentation — https://langchain-ai.github.io/langgraph/
- LangSmith — LLM Observability — https://docs.smith.langchain.com/

## Source links

- https://python.langchain.com/docs/introduction/
- https://langchain-ai.github.io/langgraph/
- https://docs.smith.langchain.com/

## FAQ

### What is the main trade-off between using LangChain and building custom LLM orchestration?

LangChain accelerates prototyping — a RAG pipeline can be up in 50 lines of Python versus 200+ lines of custom code — but every abstraction layer adds debugging distance from the underlying API. When something breaks, you get a stack trace through 5 layers of LangChain internals before seeing the actual API error, which can turn a simple bug into a multi-hour investigation.

### When should you skip LangChain and write direct API calls instead?

Skip LangChain when your workflow involves fewer than 3 LLM calls per user request, when you need precise control over token usage and caching, or when streaming output with specific UI behavior is required. In those cases, the abstraction cost outweighs the convenience, and direct calls using Anthropic's or OpenAI's official SDKs are simpler to debug and maintain.

### Is LangGraph worth using even if you avoid the rest of LangChain?

Yes — the post treats LangGraph separately from LangChain proper. Its graph-based model (nodes as actions, edges as state transitions) is genuinely useful for complex multi-step agent workflows that have branching paths, conditional execution, and state that must persist across multiple turns, even if you otherwise prefer custom orchestration.

### What does the post recommend for observability when building LLM applications?

The post recommends integrating LangSmith from day one, regardless of whether you use LangChain. LangSmith captures full trace data — every LLM call, every chain step, every token count — and its free tier is described as generous enough for development and small-scale production. Importantly, it works independently of LangChain, so it can add observability to custom-built pipelines as well.

### How should teams handle LangChain's frequent breaking changes in production?

Pin versions strictly in requirements.txt and budget dedicated time for version migration every 6–12 months. The post warns that LangChain has had numerous breaking changes between major versions and that teams have lost entire weekends to upgrade problems, so version discipline and planned migration windows are essential for production systems.

---

- HTML version: https://www.matthewswong.com/en/blog/langchain-vs-custom-llm/
- Bahasa Indonesia: https://www.matthewswong.com/id/blog/langchain-vs-custom-llm/
- Site index for agents: https://www.matthewswong.com/llms.txt
- Sitemap: https://www.matthewswong.com/sitemap.xml
- API description (OpenAPI): https://www.matthewswong.com/openapi.json