c057f8efb7
A third deployment next to the two on the DGX Spark, which stay as they
are: deploy/k8s/ has its own image and Kustomize manifests. The API is a
Deployment with probes, the data a PersistentVolumeClaim, the nightly
update a CronJob; overlays/gpu adds cuOpt on a GPU of the cluster.
Five assumptions of the single-container setup did not hold for pods, so
the service changed (on the Spark nothing changes that one would notice):
- probes: GET /health/live (async, no I/O) and GET /health/ready (data
directory readable); neither needs cuOpt or market data, so a cuOpt
outage never gets the API restarted. /health is unchanged
- DataStore replaces every file atomically (temporary file renamed into
place) and saves the manifest last; the API reloads its market data
when the manifest changes, so it serves what a CronJob wrote
- one writer at a time across processes and pods: an flock on
/data/.write.lock, taken by API jobs and the CLI's data commands
(setup, reselect, update, clean, snapshot); a second writer gets 409 or
exit status 3, with the holder's name. `clean` now updates the manifest
- job records live in /data/jobs/ and are written at every status
change: every replica answers GET /jobs/{id}, and a job whose process
has gone is reported failed ("interrupted")
- the image runs as uid 10001 under a read-only root file system, with
the library versions pinned to deploy/apptainer/constraints.txt (SciPy
1.18 releases the GIL while HiGHS solves, so the liveness probe is
answered during an optimisation)
docs/KUBERNETES.md is the guide (build, deploy, load the data, operate,
troubleshooting); README, USAGE and ARCHITECTURE point to it.
Both kustomizations render (kubectl kustomize); the k3s test of the
deployment is recorded in docs/KUBERNETES.md. 149 tests (148 pass, the
cuOpt round trip skipped without a server).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
42 lines
2.3 KiB
Docker
42 lines
2.3 KiB
Docker
# quantcore for Kubernetes (deploy/k8s/, guide: docs/KUBERNETES.md). The DGX Spark deployments keep
|
|
# their own images: deploy/Dockerfile (Docker Compose) and deploy/apptainer/quantcore.def (Apptainer).
|
|
#
|
|
# Build context is the repository root; build for the platform of the cluster's nodes:
|
|
# docker build -f deploy/k8s/Dockerfile -t quantcore:0.1.0 . # this machine's
|
|
# docker buildx build --platform linux/arm64,linux/amd64 -f deploy/k8s/Dockerfile \
|
|
# -t <registry>/quantcore:0.1.0 --push . # both, to a registry
|
|
#
|
|
# What differs from deploy/Dockerfile, all of it for running as a pod:
|
|
# * the library versions are pinned (the Apptainer image's constraints.txt): SciPy 1.18 releases
|
|
# the GIL while HiGHS solves, so the liveness probe is answered during a long optimisation
|
|
# * it runs as an unprivileged user (uid 10001) whose HOME is /tmp: under the pod's read-only root
|
|
# file system, /tmp and /data are the only writable paths (yfinance caches its cookies in HOME)
|
|
# * no HEALTHCHECK: Kubernetes ignores it; the probes are in base/deployment.yaml
|
|
FROM python:3.12-slim
|
|
|
|
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
|
PIP_ROOT_USER_ACTION=ignore QUANTCORE_DATA_DIR=/data \
|
|
OPENBLAS_NUM_THREADS=4 OMP_NUM_THREADS=4 MKL_NUM_THREADS=4
|
|
# BLAS threading: the innovation filter runs thousands of small (N x N) eigendecompositions;
|
|
# OpenBLAS with all cores is ~50x slower on those than with 1-4 threads.
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends tini \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& groupadd --system --gid 10001 quantcore \
|
|
&& useradd --system --uid 10001 --gid 10001 --home-dir /tmp --no-create-home --shell /usr/sbin/nologin quantcore \
|
|
&& mkdir /data && chown 10001:10001 /data
|
|
|
|
# the pinned libraries first: this layer survives code changes, so a rebuild only installs quantcore
|
|
COPY deploy/apptainer/constraints.txt /src/constraints.txt
|
|
RUN pip install -r /src/constraints.txt
|
|
COPY pyproject.toml README.md /src/
|
|
COPY quantcore /src/quantcore
|
|
RUN pip install -c /src/constraints.txt /src && rm -rf /src
|
|
|
|
WORKDIR /app
|
|
USER 10001:10001
|
|
EXPOSE 5001
|
|
|
|
ENTRYPOINT ["/usr/bin/tini", "--"]
|
|
CMD ["uvicorn", "quantcore.api.app:app", "--host", "0.0.0.0", "--port", "5001", "--workers", "1"]
|