fac41ffdfc
The solver benchmark (GPU_Optimization/cuopt_benchmark, November 2026) showed
that the integer mean-CVaR MILP is an LP in disguise: HiGHS closes it at the
root node, only the tail scenario rows bind, and the cardinality constraint is
the only hard combinatorics. quantcore.optimization.large_scale exploits that:
LP relaxation by tail-row generation (evaluated at the continuous point, its
objective a valid bound), screening to the top names under a cardinality limit,
a boxed round-and-repair MILP, certification against the bound with escalation
to the integer row generation otherwise. Solver choice by size (HiGHS below
16 000 rows, cuOpt above).
Request option `strategy` ('auto' | 'milp' | 'large_scale'; auto from 200
candidates or 32 768 scenarios) in the service, the API schema and both CLIs.
Settings from the benchmark: cuOpt presolve off (its asynchronous GPU heuristic
otherwise holds every MILP for 5-25 s), dual simplex for pure LPs (concurrent
returns PDLP's inexact point on the L40S), 50 ms polling. rowgen takes an
initial working set.
Measured with HiGHS (M4 Max): N=300 S=4096 5.2 s -> 0.7 s, N=300 S=16384
45 s -> 2.9 s, N=750 S=16384 15.6 s -> 7.4 s, K=40 of 750 24.6 s -> 7.3 s,
objectives within the 1e-4 gap. Tests in tests/test_large_scale.py; benchmark
script scripts/experiments/large_scale.py; docs updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Experiments behind the scenario-sampling section
These scripts produced the tables in docs/optimization_model.pdf, Section "Scenario sets:
sampling error, variance reduction and large sets". They run against the local data store
(quantcore setup first) and print their tables; they are not part of the package or the tests.
| script | what it measures |
|---|---|
scenario_noise.py N S SEEDS [cont|int] [kinds] [beta] |
GBM model: plain MC vs antithetic / moment matching / LHS / Sobol' at equal S, continuous (weights) or integer problem, against the exact Gaussian optimum (cutting planes) — Table "Sampling noise" |
empirical_noise.py N S SEEDS BETA [kinds] |
empirical-innovation model: bootstrap vs moment-matched / balanced / rank-stratified resampling, evaluated on a 500 000-draw set |
is_noise.py N SIZES SEEDS BETAS |
importance sampling (two-stage tilt, defensive mixture) vs moment-matched Sobol' — Table "Importance sampling" |
scaling.py N SIZES highs|cuopt |
solve time versus the number of scenarios (HiGHS locally / cuOpt on the Spark) — Table "Solve times" |
rowgen.py N SIZES [full] |
tail-row generation versus the full MILP |
gurobi_ref.py N BETA |
the exact Gaussian MISOCP solved directly with Gurobi (needs gurobipy and a licence large enough for N) — validation of the cutting-plane reference |
Example: python scripts/experiments/scenario_noise.py 30 2048 8 cont mc,antithetic,mc+mm,sobol,sobol+mm 0.95