b5da424dfe
The portfolio data, risk and mean-CVaR optimisation service, prepared for students who run it on their own DGX Spark without root, Docker or Tailscale. - deploy/apptainer/qc.sh: install, start/stop/status/logs, cli/api/python, data import/export and Yahoo setup/update. cuOpt and quantcore run as two Apptainer instances of the user and talk over 127.0.0.1; the API and the test bench listen on 127.0.0.1 only. Builds quantcore.sif without root, or falls back to a plain Python image plus a virtualenv when the Spark allows no unprivileged builds. Detects taken ports and suggests free ones. - deploy/apptainer/quantcore.def + constraints.txt: the image, with the library versions of the verified production container pinned. - cuOpt under Apptainer: NVIDIA's image sets HOME=/opt/cuopt, read-only in a SIF, so the solver died on its first job; qc.sh gives it a writable home. - solvers: a cuOpt that disappears during a solve now falls back to HiGHS instead of answering 500 (requests' ConnectionError escaped the fallback). - client: idempotent GETs retry a dropped keep-alive connection, which made `quantcore-api update --wait` fail when a poll met uvicorn's 5 s timeout. - config: cuOpt defaults to http://127.0.0.1:5000 and `quantcore serve` to 127.0.0.1; the Docker images keep passing their own values. - docs/SPARK_APPTAINER.md: the students' guide; README, USAGE, ARCHITECTURE and SPARK_DEPLOYMENT point to it. Tested on a DGX Spark (GB10, Apptainer 1.5.4, unprivileged user) in both modes: selftest on every endpoint, optimisation on cuOpt, Yahoo update, port conflicts; 139 tests pass in the container against the live cuOpt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
58 lines
2.1 KiB
Bash
Executable File
58 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Deploy the quant service to the DGX Spark:
|
|
# 1. rsync the repository to ~/services/work/Portfolio_Risk_Optimization
|
|
# 2. optionally rsync an already built data directory (saves the 15-minute setup on the Spark)
|
|
# 3. add the `quantcore` service block to ~/services/docker-compose.yml (idempotent)
|
|
# 4. build the image and start the container
|
|
#
|
|
# Usage: deploy/deploy_to_spark.sh [--with-data] [--host user_fh_fhgr@spark-6d4a]
|
|
set -euo pipefail
|
|
|
|
HOST="user_fh_fhgr@spark-6d4a"
|
|
WITH_DATA=0
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
--with-data) WITH_DATA=1 ;;
|
|
--host=*) HOST="${arg#--host=}" ;;
|
|
--host) shift; HOST="$1" ;;
|
|
esac
|
|
done
|
|
|
|
REPO_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
|
REMOTE_SERVICES="services"
|
|
REMOTE_REPO="$REMOTE_SERVICES/work/Portfolio_Risk_Optimization"
|
|
REMOTE_DATA="$REMOTE_SERVICES/work/quantcore_data"
|
|
|
|
echo ">> rsync repository -> $HOST:$REMOTE_REPO"
|
|
ssh "$HOST" "mkdir -p $REMOTE_REPO $REMOTE_DATA"
|
|
rsync -az --delete \
|
|
--exclude '.git' --exclude '/data/' --exclude '__pycache__' --exclude '*.pyc' --exclude '.pytest_cache' \
|
|
--exclude '*.egg-info' --exclude '.ruff_cache' --exclude 'scratch/' \
|
|
"$REPO_DIR/" "$HOST:$REMOTE_REPO/"
|
|
|
|
if [[ "$WITH_DATA" == "1" && -d "$REPO_DIR/data" ]]; then
|
|
echo ">> rsync data directory -> $HOST:$REMOTE_DATA"
|
|
rsync -az --exclude 'jobs/' --exclude '*.log' "$REPO_DIR/data/" "$HOST:$REMOTE_DATA/"
|
|
fi
|
|
|
|
echo ">> register the service in ~/services/docker-compose.yml (if missing)"
|
|
ssh "$HOST" bash -s <<'REMOTE'
|
|
set -euo pipefail
|
|
cd ~/services
|
|
if ! grep -q '^ quantcore:' docker-compose.yml; then
|
|
cp docker-compose.yml "docker-compose.yml.bak.$(date +%Y%m%d%H%M%S)"
|
|
# append the service block (everything below "services:" of the snippet)
|
|
awk 'f{print} /^services:/{f=1}' work/Portfolio_Risk_Optimization/deploy/docker-compose.quantcore.yml >> docker-compose.yml
|
|
echo " added quantcore service block"
|
|
else
|
|
echo " quantcore service already present"
|
|
fi
|
|
docker compose build quantcore
|
|
docker compose up -d quantcore
|
|
sleep 3
|
|
docker compose ps quantcore
|
|
curl -s http://localhost:5001/health | head -c 600; echo
|
|
REMOTE
|
|
|
|
echo ">> done. API: http://spark-6d4a:5001/docs"
|