# Prompt Engineering for Production: Beyond the Basics > Production prompt engineering is different from tutorial prompt engineering — version control, testing, regression prevention, and the patterns that actually improve reliability. URL: https://www.matthewswong.com/en/blog/prompt-engineering-production/ - Author: Matthews Wong (https://www.matthewswong.com/en/about/) - Published: 2025-06-10 - Updated: 2025-06-10 - Category: AI Everyone's blog post about prompt engineering covers chain-of-thought, few-shot examples, and role instructions. That's fine for demos. Production prompts have different requirements: they need to be versioned, tested, monitored for regression, and optimized for cost. ## Treating Prompts as Code: Version Control The biggest mistake I made early on was storing prompts inline in application code. The fix: treat prompts as artifacts with their own versioning. I store prompts in a prompts/ directory as Markdown files with YAML frontmatter containing version, model, max_tokens, temperature, and a description. ### Prompt Templating and Variable Injection Use a proper template library: Handlebars (JavaScript) or Jinja2 (Python) give you partials (reusable prompt components), conditional sections, and loop constructs. Extract common prompt components into shared partials that all prompts import. ### Few-Shot Example Management Few-shot examples are some of the most powerful prompt engineering tools and some of the hardest to manage. Store examples in a separate examples/ directory, tagged with the prompt they belong to and a validity date. Review and refresh examples quarterly. > The most reliable prompt engineering pattern I've found for structured output is providing the output format as a complete example in the prompt, not as a JSON Schema description. Models follow examples better than schemas, and you catch format violations before they reach your JSON parser. ## Prompt Testing: Regression Prevention The most common production prompt problem is regression: a prompt change that improves behavior on the failing case you were fixing degrades behavior on previously-working cases. I maintain an evals/ directory with test cases and run them on every prompt change using a CI step. ### Automatic Prompt Optimization Once you have a test suite, you can use automatic prompt optimization: tools like DSPy (Stanford) that automatically generate and test prompt variations. I used DSPy to optimize my workout recommendation prompt and got a 22% improvement in eval score. > **Temperature 0 Is Not Always What You Want** > Setting temperature to 0 is the common advice for production prompts. But temperature 0 can cause quality degradation: it amplifies small prompt changes and produces overconfident outputs. I use temperature 0.1 as my default for production prompts — nearly deterministic, but with enough variance to avoid the deterministic failure modes. ## Cost Optimization Through Prompt Engineering Better prompts are often shorter prompts. I ran a prompt audit on my AI Gymbro system prompt and reduced it from 1,100 tokens to 620 tokens with no measurable quality degradation. That's 480 fewer tokens per request — at 10,000 requests/day, it's 4.8M tokens/day saved. ## Model-Specific Prompt Differences Prompts don't transfer perfectly between models. Claude responds better to explicit step-by-step thinking instructions. GPT-4o tends to follow format instructions more literally. Gemini models respond well to explicit output examples. When you migrate a prompt from one model to another, budget time for re-optimization. ## Building a Prompt Ops Practice Long-term, managing prompts at scale requires: version control (Git), testing (evals), CI/CD (automatic eval runs on changes), observability (log prompt versions alongside LLM calls), and a change review process. I use a 'prompt PR' process for all prompt changes. ## Sources & Further Reading - Stanford NLP — DSPy: Compiling Declarative Language Model Calls — https://github.com/stanfordnlp/dspy - PromptFoo — Prompt Testing and Evaluation — https://promptfoo.dev/docs/intro - Anthropic — Prompt Engineering Guide — https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview ## Source links - https://github.com/stanfordnlp/dspy - https://promptfoo.dev/docs/intro - https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview ## FAQ ### Why should production prompts be version-controlled separately from application code? Storing prompts inline as string templates buried in service functions makes them hard to track, review, and roll back. Treating prompts as artifacts in a dedicated prompts/ directory with YAML frontmatter (version, model, max_tokens, temperature) gives them the same auditability as any other code change. This also enables a 'prompt PR' review process so changes are deliberate and visible. ### What is prompt regression and how do you prevent it? Prompt regression happens when a change that fixes one failing case silently breaks previously-working cases. The solution is maintaining an evals/ directory of test cases and running them on every prompt change as a CI step, so regressions are caught before they reach production. This mirrors how software tests prevent code regressions. ### Why does the post recommend temperature 0.1 instead of temperature 0 for production? Setting temperature to 0 amplifies the effect of small prompt changes and can produce overconfident outputs, which are harder to debug. A value of 0.1 is nearly deterministic but introduces just enough variance to avoid these deterministic failure modes, making it a safer default for production prompts. ### How did prompt optimization reduce token costs in the AI Gymbro system prompt? A prompt audit on the AI Gymbro system prompt cut it from 1,100 tokens down to 620 tokens with no measurable quality degradation — a reduction of 480 tokens per request. At 10,000 requests per day, that translates to 4.8 million tokens saved daily, demonstrating that better prompts are often shorter prompts. ### Can prompts be reused across different LLM models without changes? No — prompts do not transfer perfectly between models. Claude responds better to explicit step-by-step thinking instructions, GPT-4o tends to follow format instructions more literally, and Gemini models respond well to explicit output examples. When migrating a prompt from one model to another, budget dedicated time for re-optimization. --- - HTML version: https://www.matthewswong.com/en/blog/prompt-engineering-production/ - Bahasa Indonesia: https://www.matthewswong.com/id/blog/prompt-engineering-production/ - Site index for agents: https://www.matthewswong.com/llms.txt - Sitemap: https://www.matthewswong.com/sitemap.xml - API description (OpenAPI): https://www.matthewswong.com/openapi.json