c057f8efb7
A third deployment next to the two on the DGX Spark, which stay as they
are: deploy/k8s/ has its own image and Kustomize manifests. The API is a
Deployment with probes, the data a PersistentVolumeClaim, the nightly
update a CronJob; overlays/gpu adds cuOpt on a GPU of the cluster.
Five assumptions of the single-container setup did not hold for pods, so
the service changed (on the Spark nothing changes that one would notice):
- probes: GET /health/live (async, no I/O) and GET /health/ready (data
directory readable); neither needs cuOpt or market data, so a cuOpt
outage never gets the API restarted. /health is unchanged
- DataStore replaces every file atomically (temporary file renamed into
place) and saves the manifest last; the API reloads its market data
when the manifest changes, so it serves what a CronJob wrote
- one writer at a time across processes and pods: an flock on
/data/.write.lock, taken by API jobs and the CLI's data commands
(setup, reselect, update, clean, snapshot); a second writer gets 409 or
exit status 3, with the holder's name. `clean` now updates the manifest
- job records live in /data/jobs/ and are written at every status
change: every replica answers GET /jobs/{id}, and a job whose process
has gone is reported failed ("interrupted")
- the image runs as uid 10001 under a read-only root file system, with
the library versions pinned to deploy/apptainer/constraints.txt (SciPy
1.18 releases the GIL while HiGHS solves, so the liveness probe is
answered during an optimisation)
docs/KUBERNETES.md is the guide (build, deploy, load the data, operate,
troubleshooting); README, USAGE and ARCHITECTURE point to it.
Both kustomizations render (kubectl kustomize); the k3s test of the
deployment is recorded in docs/KUBERNETES.md. 149 tests (148 pass, the
cuOpt round trip skipped without a server).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
24 lines
759 B
YAML
24 lines
759 B
YAML
# quantcore on Kubernetes, CPU only: kubectl apply -k deploy/k8s/base (guide: docs/KUBERNETES.md)
|
|
#
|
|
# Runs on any cluster; every optimisation uses HiGHS inside the API pod. overlays/gpu adds cuOpt on
|
|
# an NVIDIA GPU of the cluster; a cuOpt that runs elsewhere is one line in quantcore.env.
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
namespace: quantcore
|
|
labels:
|
|
- pairs:
|
|
app.kubernetes.io/part-of: quantcore
|
|
resources:
|
|
- namespace.yaml
|
|
- pvc.yaml
|
|
- deployment.yaml
|
|
- service.yaml
|
|
- cronjobs.yaml
|
|
configMapGenerator:
|
|
- name: quantcore-config
|
|
envs:
|
|
- quantcore.env
|
|
images:
|
|
- name: quantcore # built with deploy/k8s/Dockerfile; a registry: add newName: <registry>/quantcore
|
|
newTag: "0.1.0"
|