DevOps
k6 SLO Quality Gate for Bitbucket Pipelines CI/CD
July 20269 min read
Photo by ChrisO via Openverse (CC BY-SA)
An SLO quality gate is an automated pipeline step that runs a short performance test against a freshly deployed build and fails the pipeline if the results breach a defined service-level objective, such as a maximum p95 response time or error rate. Instead of relying on someone remembering to test performance manually, the pipeline itself blocks a bad build from being promoted further. It turns a manual, easily-skipped check into a step that runs on every single deploy automatically.
A full comparative load test can run for minutes with many virtual users and is meant to characterise performance under stress, which is too slow and too resource-heavy to run on every deploy. A five-virtual-user, two-minute smoke test is fast enough to fit inside a normal release cycle while still being enough traffic to catch a clearly broken endpoint or a major latency regression. The full load-testing methodology stays a separate, manually-triggered process for deeper analysis.
k6 evaluates its configured thresholds at the end of the run and exits with a non-zero process code if any threshold is breached. Bitbucket Pipelines treats a non-zero exit code as a failing step and stops the custom pipeline there, which means the manual Deploy to Production stage that normally follows never becomes available. The failing build is blocked from reaching production without anyone having to notice the regression by hand.
k6 lets a script define pass and fail criteria directly in its options export under a thresholds key, for example http_req_duration set to fail once the 95th percentile reaches a given millisecond value, and a custom error rate metric set to fail once it reaches a given percentage. k6 tracks every request against these rules throughout the run and applies the verdict automatically when the test finishes, with no separate assertion script required.
In this setup the gate adds roughly two minutes to each staging deploy cycle: about thirty seconds waiting for the service to finish starting up, plus two minutes of smoke-test traffic. That is a real recurring cost on every release, but it is small compared to the cost of a performance regression reaching production undetected, which previously depended entirely on someone remembering to test manually.
Photo by ChrisO via Openverse (CC BY-SA)
Key Takeaway
This post explains how to turn an existing Bitbucket Pipelines release flow into an automated performance gate by appending one k6 smoke-test step right after the staging deploy. The step checks p95 latency and error-rate thresholds, posts pass or fail annotations to Grafana, and blocks the manual production-promotion stage automatically whenever a regression crosses those limits.
During my SGU thesis work at Commsult Indonesia on Ontego Traces, a Java Spring Boot logistics platform running on Docker Swarm, I found the release pipeline had a load-testing framework, an auto-scaler, and a full observability stack, but the actual path from a merged commit to a production deploy still had zero performance validation. A slow query or a lock-contention regression could sail through code review, pass the Java test suite, and land in production, and the first person to notice would be a customer complaining about a slow screen. This post covers the fourth pillar of that thesis project: turning the existing Bitbucket Pipelines stage-release flow into an automated gate that catches those regressions before a human ever clicks Deploy to Production.
The stage-release pipeline already did a lot right. It built the Maven project, pushed a versioned Docker image to the registry, and used Ansible to deploy the new image to a staging stack. What it did not do was ask the one question that matters most before a production promotion: does this build still meet its latency and error-rate targets? Performance validation was entirely manual. Someone had to remember to open a terminal, run a k6 script by hand, read the numbers, and judge whether they looked acceptable. In practice that step got skipped under deadline pressure, and a regression that increased p95 latency or introduced a failing endpoint could reach the manual Deploy to Production stage completely undetected. The other pillars of the thesis project, the k6 load-testing scripts, the CPU-based auto-scaler, and the Grafana observability stack, existed and worked well on their own, but nothing wired them into the one moment where a bad build could still be stopped: the gap between a staging deploy and a production promotion.
Rather than build a separate performance-testing pipeline that someone would also have to remember to trigger, I appended a single step to the existing stage-release custom pipeline, directly after the Ansible deploy-to-staging step and directly before the manual Deploy to Production stage. The new step runs immediately after every staging deploy, with no separate trigger and no extra click required from anyone. It runs a k6 smoke test, five virtual users for two minutes, deliberately lighter than the ten-VU, 120-second comparative load profile used for the benchmarking side of the thesis, against the just-deployed staging stack, and it exits non-zero if the response time or error rate crosses a configured threshold. Because Bitbucket Pipelines stops a custom pipeline at the first failing step, a non-zero exit here automatically disables the manual production-promotion stage that follows. There is no code path left for a failing build to reach a human standing in front of a Deploy to Production button.
k6 exposes pass and fail criteria as a first-class thresholds block inside the script's own options export, so the gate logic lives in the same file as the load profile instead of a separate assertion script bolted on afterward. The gate treats two conditions as a failure: the 95th percentile of http_req_duration reaching or exceeding 5000 milliseconds, and the request error rate reaching or exceeding 5 percent. Both numbers are deliberately lenient for a staging environment running on a single replica with cold caches. The goal at this stage is to catch clear regressions and outright breakage, not to enforce the tighter latency budget the production SLO eventually needs. k6 evaluates every threshold automatically at the end of the run and sets its own process exit code accordingly, which is exactly the signal Bitbucket Pipelines needs to fail the step and stop the pipeline.
// k6/ci-smoke-test.js — thresholds live next to the load profile
export const options = {
stages: [
{ duration: "20s", target: 5 }, // ramp up
{ duration: "90s", target: 5 }, // steady load
{ duration: "10s", target: 0 }, // ramp down
],
thresholds: {
http_req_duration: ["p(95)<5000"], // fail if p95 >= 5000ms
error_rate: ["rate<0.05"], // fail if error rate >= 5%
},
};
Keep the CI threshold looser than your eventual production SLO on purpose. This gate runs against a single-replica staging stack with cold caches, so a strict production-grade p95 target would fail on almost every deploy for reasons that have nothing to do with the code you just shipped. Tighten the threshold only after the underlying application-level bottleneck is actually fixed, not before.
The step runs on the official k6 Docker image, so no separate binary install is needed on the runner. It waits roughly thirty seconds for the freshly deployed Spring Boot service to finish starting up, then runs the smoke test script and captures its exit code in a variable before doing anything else with it. Capturing the exit code first, instead of letting the k6 command be the final line of the step, matters because it lets the step post a closing Grafana annotation and still propagate the original pass or fail result to Bitbucket Pipelines afterward.
# bitbucket-pipelines.yml — appended after "Deploy to Stage",
# directly before the manual "Deploy to Production" stage
- step:
name: "Performance Quality Gate (k6)"
image:
name: grafana/k6:0.49.0
script:
# Give the freshly deployed service time to finish startup
- sleep 30
# Mark the run boundary on the shared Grafana dashboard
- |
curl -s -X POST "${GRAFANA_URL}/api/annotations" \
-H "Authorization: Bearer ${GRAFANA_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"tags\":[\"cicd\",\"pipeline-start\"],
\"text\":\"CI pipeline #${BITBUCKET_BUILD_NUMBER} started\"}"
# Run the smoke test against the just-deployed staging stack
- |
k6 run \
--out influxdb=http://${INFLUXDB_URL}/k6 \
-e PIPELINE_ID=${BITBUCKET_BUILD_NUMBER} \
k6/ci-smoke-test.js
- GATE_EXIT=$?
# Mark pass/fail on the same dashboard, then propagate the result
- |
RESULT_TAG=$([ $GATE_EXIT -eq 0 ] && echo "pass" || echo "fail")
curl -s -X POST "${GRAFANA_URL}/api/annotations" \
-H "Authorization: Bearer ${GRAFANA_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"tags\":[\"cicd\",\"pipeline-end\",\"${RESULT_TAG}\"],
\"text\":\"CI #${BITBUCKET_BUILD_NUMBER} finished\"}"
- exit $GATE_EXIT
The smoke test streams every request straight to InfluxDB through k6's built-in InfluxDB output flag, the same time-series store that the thesis's live observability stack already writes container and JVM metrics to. That means a gate run does not produce a separate report someone has to go dig up after the fact. Its latency and error-rate metrics land on the identical Grafana dashboard an operator already has open to watch production traffic. The pipeline step also calls the Grafana HTTP API twice, once right before the k6 run starts and once right after it finishes, to write annotations marking the start and end of the gate on that same dashboard's time axis, tagged pass or fail. A failed gate is not a build log someone has to go read after the fact. It is a red marker on the dashboard the team is already watching, sitting at the exact timestamp the regression was caught.
Capture the k6 exit code into a variable before calling exit, and post the closing annotation in between. If the exit happens on the same line as the k6 invocation, a failing run skips the second annotation call entirely, and the dashboard never records that the gate failed, which defeats the entire point of putting the marker there in the first place.
The gate step adds roughly two minutes to every staging deploy cycle: thirty seconds waiting for the service to finish starting, then two minutes of k6 traffic against it. That is a real, recurring cost paid on every single release, whether or not that particular build happens to introduce a regression. In exchange, it automatically blocks the exact class of regression that previously depended entirely on someone remembering to test by hand. A slow query, a newly introduced N+1 pattern, or a broken endpoint that used to reach production undetected now fails the build before a human ever sees the manual production-promotion button. Two other posts from this same thesis project cover the load-testing methodology behind the k6 script itself and the auto-scaler that reacts to the traffic this gate generates; this post is only about the automation step that ties a pass or fail decision to the release pipeline.