strategy='auto' now routes every scenario-mode CVaR optimisation through
quantcore.optimization.large_scale (LP relaxation on the tail rows, screening,
round-and-repair, certified against the relaxation bound, escalation to the
integer MILP otherwise); strategy='milp' is the explicit opt-out, and
row_generation applies to that path only. When the escalated MILP proves the
rounded portfolio optimal within its gap, the result is Optimal even where the
relaxation bound is loose (small portfolios with a real integrality gap).
UI: a Strategy selector and the driver's bound, gap, screened size, rows and
solver calls in the provenance card. Deploy env examples document the new
cuOpt settings (presolve off, dual simplex for LPs, 50 ms polling). The
service-equivalence test asks for strategy='milp' where it checks the
integer row-generation path.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The solver benchmark (GPU_Optimization/cuopt_benchmark, November 2026) showed
that the integer mean-CVaR MILP is an LP in disguise: HiGHS closes it at the
root node, only the tail scenario rows bind, and the cardinality constraint is
the only hard combinatorics. quantcore.optimization.large_scale exploits that:
LP relaxation by tail-row generation (evaluated at the continuous point, its
objective a valid bound), screening to the top names under a cardinality limit,
a boxed round-and-repair MILP, certification against the bound with escalation
to the integer row generation otherwise. Solver choice by size (HiGHS below
16 000 rows, cuOpt above).
Request option `strategy` ('auto' | 'milp' | 'large_scale'; auto from 200
candidates or 32 768 scenarios) in the service, the API schema and both CLIs.
Settings from the benchmark: cuOpt presolve off (its asynchronous GPU heuristic
otherwise holds every MILP for 5-25 s), dual simplex for pure LPs (concurrent
returns PDLP's inexact point on the L40S), 50 ms polling. rowgen takes an
initial working set.
Measured with HiGHS (M4 Max): N=300 S=4096 5.2 s -> 0.7 s, N=300 S=16384
45 s -> 2.9 s, N=750 S=16384 15.6 s -> 7.4 s, K=40 of 750 24.6 s -> 7.3 s,
objectives within the 1e-4 gap. Tests in tests/test_large_scale.py; benchmark
script scripts/experiments/large_scale.py; docs updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The portfolio data, risk and mean-CVaR optimisation service, prepared for
students who run it on their own DGX Spark without root, Docker or Tailscale.
- deploy/apptainer/qc.sh: install, start/stop/status/logs, cli/api/python,
data import/export and Yahoo setup/update. cuOpt and quantcore run as two
Apptainer instances of the user and talk over 127.0.0.1; the API and the
test bench listen on 127.0.0.1 only. Builds quantcore.sif without root, or
falls back to a plain Python image plus a virtualenv when the Spark allows
no unprivileged builds. Detects taken ports and suggests free ones.
- deploy/apptainer/quantcore.def + constraints.txt: the image, with the
library versions of the verified production container pinned.
- cuOpt under Apptainer: NVIDIA's image sets HOME=/opt/cuopt, read-only in a
SIF, so the solver died on its first job; qc.sh gives it a writable home.
- solvers: a cuOpt that disappears during a solve now falls back to HiGHS
instead of answering 500 (requests' ConnectionError escaped the fallback).
- client: idempotent GETs retry a dropped keep-alive connection, which made
`quantcore-api update --wait` fail when a poll met uvicorn's 5 s timeout.
- config: cuOpt defaults to http://127.0.0.1:5000 and `quantcore serve` to
127.0.0.1; the Docker images keep passing their own values.
- docs/SPARK_APPTAINER.md: the students' guide; README, USAGE, ARCHITECTURE
and SPARK_DEPLOYMENT point to it.
Tested on a DGX Spark (GB10, Apptainer 1.5.4, unprivileged user) in both
modes: selftest on every endpoint, optimisation on cuOpt, Yahoo update,
port conflicts; 139 tests pass in the container against the live cuOpt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
