Kafka: 2. The lifecycle script
This documentation is part of the GitHub Actions & GitLab CI guide. View the full guide here: Spin up a real Kafka service from your GitHub Actions or GitLab CI pipeline, run your tests against it, and tear it down automatically.
👋 Welcome to the Stackhero documentation!
Stackhero gives you a fully managed Kafka cloud environment, ready in minutes. Here is what you can expect:
- No limits on message size or transfer: scale your data without worrying about caps.
- Simple, one-click updates keep your platform current without downtime or manual work.
- Consistent high performance and robust security on your own private, dedicated infrastructure.
If you want to move faster and reduce operational overhead, Stackhero provides Kafka cloud hosting to help you. You can have a production-ready cluster in about 5 minutes.
Here is a complete example script that manages the full service lifecycle. It demonstrates how little setup is required, just a few commands. You can copy the ready-made YAML for your platform from the sections below.
#!/bin/bash
set -euo pipefail
# STACKHERO_TOKEN is supplied by the CI secret, the CLI picks it up automatically.
stackName="ci-kafka-$$" # Unique stack name for each run
serviceStore="kafka" # The Kafka service store
instance="10G" # Change this as needed (see step 3)
region="europe" # Region name (see stackhero regions-list)
# 1. Install the CLI on the runner
curl -fsSL https://www.stackhero.io/install.sh | sh
# 2. Install the client required for smoke testing (jq + curl are the baseline)
apt-get update && apt-get install -y --no-install-recommends jq curl netcat-openbsd
# 3. Create a dedicated stack for this run
stackId=$(stackhero --format=script stack-create --name="$stackName")
echo "Stack created: $stackId"
# 4. Add Kafka and capture its service id
serviceId=$(stackhero --format=script service-add \
--stack="$stackId" \
--service-store="$serviceStore" \
--instance="$instance" \
--region="$region")
echo "Service added: $serviceId"
# 5. Wait until the service is running (this may take a couple of minutes)
stackhero service-wait-for --service="$serviceId"
# 6. Read the configuration (contains the generated credentials)
config=$(stackhero service-configuration-get --service="$serviceId" --format=json)
# 7. Extract the credentials you need
host=$(echo "$config" | jq -r '.configuration.domain')
# 8. Smoke test: Open a TCP connection to the broker port.
nc -z -w 5 "$host" 9092
echo "✅ Kafka is reachable from CI."
The teardown step, which deletes the service, waits for its removal, and then deletes the stack, is handled in the platform-specific sections below. This approach makes sure cleanup always happens, even if a smoke test fails.
A stack can only be deleted once it is empty. Always delete the service first and wait for its removal, then delete the stack. The
service-wait-forcommand ensures the service is either running or deleted before proceeding, making it the right tool for deletion waits too.