A third deployment next to the two on the DGX Spark, which stay as they
are: deploy/k8s/ has its own image and Kustomize manifests. The API is a
Deployment with probes, the data a PersistentVolumeClaim, the nightly
update a CronJob; overlays/gpu adds cuOpt on a GPU of the cluster.
Five assumptions of the single-container setup did not hold for pods, so
the service changed (on the Spark nothing changes that one would notice):
- probes: GET /health/live (async, no I/O) and GET /health/ready (data
directory readable); neither needs cuOpt or market data, so a cuOpt
outage never gets the API restarted. /health is unchanged
- DataStore replaces every file atomically (temporary file renamed into
place) and saves the manifest last; the API reloads its market data
when the manifest changes, so it serves what a CronJob wrote
- one writer at a time across processes and pods: an flock on
/data/.write.lock, taken by API jobs and the CLI's data commands
(setup, reselect, update, clean, snapshot); a second writer gets 409 or
exit status 3, with the holder's name. `clean` now updates the manifest
- job records live in /data/jobs/ and are written at every status
change: every replica answers GET /jobs/{id}, and a job whose process
has gone is reported failed ("interrupted")
- the image runs as uid 10001 under a read-only root file system, with
the library versions pinned to deploy/apptainer/constraints.txt (SciPy
1.18 releases the GIL while HiGHS solves, so the liveness probe is
answered during an optimisation)
docs/KUBERNETES.md is the guide (build, deploy, load the data, operate,
troubleshooting); README, USAGE and ARCHITECTURE point to it.
Both kustomizations render (kubectl kustomize); the k3s test of the
deployment is recorded in docs/KUBERNETES.md. 149 tests (148 pass, the
cuOpt round trip skipped without a server).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The portfolio data, risk and mean-CVaR optimisation service, prepared for
students who run it on their own DGX Spark without root, Docker or Tailscale.
- deploy/apptainer/qc.sh: install, start/stop/status/logs, cli/api/python,
data import/export and Yahoo setup/update. cuOpt and quantcore run as two
Apptainer instances of the user and talk over 127.0.0.1; the API and the
test bench listen on 127.0.0.1 only. Builds quantcore.sif without root, or
falls back to a plain Python image plus a virtualenv when the Spark allows
no unprivileged builds. Detects taken ports and suggests free ones.
- deploy/apptainer/quantcore.def + constraints.txt: the image, with the
library versions of the verified production container pinned.
- cuOpt under Apptainer: NVIDIA's image sets HOME=/opt/cuopt, read-only in a
SIF, so the solver died on its first job; qc.sh gives it a writable home.
- solvers: a cuOpt that disappears during a solve now falls back to HiGHS
instead of answering 500 (requests' ConnectionError escaped the fallback).
- client: idempotent GETs retry a dropped keep-alive connection, which made
`quantcore-api update --wait` fail when a poll met uvicorn's 5 s timeout.
- config: cuOpt defaults to http://127.0.0.1:5000 and `quantcore serve` to
127.0.0.1; the Docker images keep passing their own values.
- docs/SPARK_APPTAINER.md: the students' guide; README, USAGE, ARCHITECTURE
and SPARK_DEPLOYMENT point to it.
Tested on a DGX Spark (GB10, Apptainer 1.5.4, unprivileged user) in both
modes: selftest on every endpoint, optimisation on cuOpt, Yahoo update,
port conflicts; 139 tests pass in the container against the live cuOpt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
