Repository navigation
feat(deploy): add a Kustomize base and a guide for Kubernetes - #699
Merged
Merged
Conversation
`deploy/kubernetes` holds one StatefulSet (one replica, one 2 GiB state volume) and one headless Service for the metrics listener. The pod runs as the image's `shepherd` user with a read-only root file system and no capabilities, so it agrees with the `restricted` Pod Security level. It has HTTP probes on `/healthz` and `/readyz`, scrape annotations, and a termination grace period of 180 seconds, which is longer than the engine's drain. The base does not create the engine config ConfigMap or the RPC URL Secret. `docs/deployment/kubernetes.md` shows how to make them, how to pin the image and the StorageClass in an overlay, and what was tested. Tested on a local kind cluster (Kubernetes 1.36) in a namespace that enforces the `restricted` level, with an image built from this tree and a mainnet node: both modules became alive, the endpoints answered, and after a pod deletion the engine stopped cleanly and the new pod opened each event trigger at its cursor. A network volume and a default-deny network policy were not tested. AI Assistance: Claude Code used for the manifests, the guide and the test run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The operator docs cover a run from source and Docker Compose. This adds the third common way to run the engine: a Kubernetes cluster, from the published image.
What changed
deploy/kubernetes/is a Kustomize base with two objects: a StatefulSet (one replica, one 2 GiB state volume) and a headless Service for the metrics listener.shepherduser (id 999) with a read-only root file system, no capabilities and no service account token, so it agrees with therestrictedPod Security level. It has HTTP probes on/healthzand/readyz, theprometheus.io/*scrape annotations, and a termination grace period of 180 seconds, which is longer than the engine's 150 second drain.docs/deployment/kubernetes.mdshows how to make them fromengine.docker.toml, how to pin the image and the StorageClass in an overlay, how to read metrics, and how to stop and upgrade.README.mdlists the new directory and the new guide.No Rust code changes.
Verification
The base was applied to a local kind cluster (Kubernetes 1.36), in a namespace with
pod-security.kubernetes.io/enforce=restricted, with an image built from the tree of #696 and a mainnet node:1/1 Running./readyzansweredready: truewith both modulesalive, and/metricshadnexum_runtime_start_time_seconds.local-store.redb.kubectl delete pod, the engine wrotegraceful shutdown complete. The new pod opened each event trigger at its cursor and did no backfill.kubectl kustomize, with the image pin and the StorageClass patch, against this branch by commit.Worth a second look
useradd -rin theDockerfile, which does not name an id. The manifest names 999 because Kubernetes refusesrunAsNonRootwith a non-numeric image user. A later change could set the id explicitly in theDockerfileso that the two cannot drift.AI Assistance: Claude Code used for the manifests, the guide and the test run.