herzogflorian b5da424dfe quantcore: standalone Apptainer setup for students' DGX Sparks
The portfolio data, risk and mean-CVaR optimisation service, prepared for
students who run it on their own DGX Spark without root, Docker or Tailscale.

- deploy/apptainer/qc.sh: install, start/stop/status/logs, cli/api/python,
  data import/export and Yahoo setup/update. cuOpt and quantcore run as two
  Apptainer instances of the user and talk over 127.0.0.1; the API and the
  test bench listen on 127.0.0.1 only. Builds quantcore.sif without root, or
  falls back to a plain Python image plus a virtualenv when the Spark allows
  no unprivileged builds. Detects taken ports and suggests free ones.
- deploy/apptainer/quantcore.def + constraints.txt: the image, with the
  library versions of the verified production container pinned.
- cuOpt under Apptainer: NVIDIA's image sets HOME=/opt/cuopt, read-only in a
  SIF, so the solver died on its first job; qc.sh gives it a writable home.
- solvers: a cuOpt that disappears during a solve now falls back to HiGHS
  instead of answering 500 (requests' ConnectionError escaped the fallback).
- client: idempotent GETs retry a dropped keep-alive connection, which made
  `quantcore-api update --wait` fail when a poll met uvicorn's 5 s timeout.
- config: cuOpt defaults to http://127.0.0.1:5000 and `quantcore serve` to
  127.0.0.1; the Docker images keep passing their own values.
- docs/SPARK_APPTAINER.md: the students' guide; README, USAGE, ARCHITECTURE
  and SPARK_DEPLOYMENT point to it.

Tested on a DGX Spark (GB10, Apptainer 1.5.4, unprivileged user) in both
modes: selftest on every endpoint, optimisation on cuOpt, Yahoo update,
port conflicts; 139 tests pass in the container against the live cuOpt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:55:28 +02:00

19 KiB
Raw Blame History

Using quantcore

quantcore is the deterministic quant core your advisory platform builds on: market data, risk and performance measurement, stock-price models, Black–Litterman expected returns and mean–CVaR portfolio optimisation in whole shares.

It contains no LLM call and makes no decisions. It computes numbers from data and from the inputs you hand it, and every answer carries the provenance of those inputs. Your platform supplies the judgement — which news matters, how strongly to believe it, what to tell the client — and calls in here for every figure it quotes.

You never have to modify this package. Treat it as a service with a contract.


1. Two ways to use it

HTTP API Python library
you need a URL the package and the data directory on the same machine
import nothing — requests, or quantcore.client from quantcore.measurement.risk import RiskService
language any Python
good for the normal case: your platform calls a running service notebooks, experiments, running your own backtest loop

Start with the API. Everything in this guide is available both ways, and the library classes are documented in README.md and in docs/ARCHITECTURE.md if you need them.

2. Install

git clone https://gitea.fhgr.ch/herzogfloria/Portfolio_Risk_Management.git
cd Portfolio_Risk_Management
pip install -e .                 # Python 3.11+

On a DGX Spark you do not need this step to run the service: deploy/apptainer/qc.sh runs it in containers (see §3 and SPARK_APPTAINER.md). Install the package only where your own code imports it, for example for quantcore.client.

This gives you two commands:

command what it is
quantcore runs the services in this process; needs the data directory locally
quantcore-api talks to a running service over HTTP; needs only its URL

3. Point at a service, or run your own

On your DGX Spark (the usual case): the service and cuOpt run as Apptainer containers of your user, reachable at 127.0.0.1 on the Spark only. SPARK_APPTAINER.md has the details:

deploy/apptainer/qc.sh install && deploy/apptainer/qc.sh data-import quantcore_data.tar.gz
deploy/apptainer/qc.sh start                 # API at http://127.0.0.1:5001 (the client's default URL)
deploy/apptainer/qc.sh api health

Someone else runs it: point the client at that machine.

export QUANTCORE_API_URL=http://<host>:5001
quantcore-api health

You run it yourself. The data has to be built once, which downloads about 1 100 tickers from Yahoo Finance and takes ~15 minutes:

quantcore setup                  # once
quantcore serve                  # HTTP API on :5001, interactive docs at /docs

Or as a container, which is how it runs in production:

docker build -f deploy/Dockerfile -t quantcore:0.1 .
docker run -p 5001:5001 -v $PWD/data:/data \
  -e QUANTCORE_CUOPT_URL=http://<your-cuopt-host>:5000 quantcore:0.1

Configuration is environment variables with a QUANTCORE_ prefix (see .env.example); the ones you are likely to touch:

variable default meaning
QUANTCORE_API_URL http://127.0.0.1:5001 which service quantcore-api and QuantcoreClient talk to
QUANTCORE_DATA_DIR ./data where the prices and the universe live
QUANTCORE_CUOPT_URL http://127.0.0.1:5000 the GPU MILP solver (a cuOpt container on the same machine); unreachable is fine, see §9
QUANTCORE_BASE_CURRENCY CHF reporting currency of every figure
QUANTCORE_ENABLE_UI true false serves the JSON API and /docs only

4. Check that it works

quantcore-api selftest

This calls every endpoint with a small request and prints what each one answered:

                                       endpoint   result   ms                                 evidence
0                                   GET /health       ok  260    745 stocks, last 2026-09-21, cuOpt up
5                                   GET /prices       ok   28                   5667 dates x 3 symbols
9                             POST /risk/assets       ok   10               3 rows, cvar95_hist 0.1259
21             POST /optimize (gaussian_exact)       ok   63  Optimal on highs, 7 positions, CVaR 44,551
...
24 ok, 0 failed, 3 skipped in 1.0 s against http://127.0.0.1:8765

Exit code 0 means the deployment is sound. Run this first whenever something behaves oddly — it separates "the service is broken" from "my request is wrong" in one second. The three skipped probes write to the store or download data; --write includes them. --full optimises on a real scenario set instead of the fast Gaussian mode.

quantcore-api endpoints lists the API surface, and <url>/docs is the interactive reference with every field of every request. That is the authoritative contract — this guide is the tour.

5. Conventions you have to know

These are not negotiable per request; they are how the whole package thinks.

  • Returns are natural logarithms. ln(P_t / P_{t-1}). A CHF return of a foreign stock is the local log-return plus the FX log-return — they add.
  • Money is CHF, always, whatever the listing currency.
  • Positions are integer share numbers, never percentages. That is the point of the MILP: you cannot buy 0.3 of a share, and at CHF 1 m the difference matters.
  • A horizon is "day", "week", "month", "quarter", "year", or an integer number of periods of the chosen frequency. "week" at daily frequency is 5 trading days.
  • Risk is CVaR (expected shortfall) at 95 % and 99 %, expressed as an absolute amount over the horizon. VaR comes along for reference.
  • Two knobs on every parametric risk figure, and the defaults are not always what you want:
    • drift: "estimated" subtracts the window's mean return; "zero" uses volatility only. Use "zero" for anything longer than a week — over a quarter the estimated mean is the noisiest input in the calculation and it can halve the reported risk.
    • space: "log" reports the figure as a log-return; "value" as a fraction of the position's value (1 - e^{-x}), which cannot exceed 100 %. Use "value" in anything a client reads.

6. The endpoints

method path what it answers
GET /health is the service up, what data does it hold, is the GPU solver reachable
GET /universe, /universe/{symbol} the investable stocks and their metadata
GET /data/symbols the universe as one table plus filter facets — feeds a stock picker
GET /prices, /returns adjusted closes and log-returns, daily/weekly/monthly, CHF or local
GET /market/index, /market/weights the cap-weighted broad market and its weights
POST /risk/assets per-stock volatility, VaR, CVaR (normal and historical, rolling and EWM)
POST /risk/series those figures through time, for a chart
POST /risk/covariance, /risk/correlation the matrices as measured
POST /risk/portfolio VaR/CVaR of a portfolio held in shares
POST /performance/assets, /performance/rolling, /performance/portfolio return, Sharpe, Sortino, drawdown, Calmar, skew, kurtosis
POST /model/expected-returns Black–Litterman: equilibrium π, your views, the posterior
POST /model/covariance the covariance the models use, with its conditioning
POST /model/scenarios a scenario set, its moments, a portfolio's P&L distribution
POST /optimize the mean–CVaR MILP: a new portfolio or a rebalancing
POST /jobs/setup, /jobs/update, GET /jobs, /jobs/{id} background data work
GET/POST /snapshots freeze the data so a demo stays reproducible

7. From Python

from quantcore.client import QuantcoreClient

qc = QuantcoreClient("http://127.0.0.1:5001")      # or omit: $QUANTCORE_API_URL, else this default

qc.health()["store"]["last_price_date"]            # '2026-09-21'

# what can I invest in?
swiss = qc.universe(country="Switzerland")
health = [s["symbol"] for s in qc.universe(region="Europe", sector="Health Care")]

# what is one stock's risk over a week, as a client would read it?
row = qc.asset_risk(["NESN.SW"], horizon="week", drift="zero", space="value")[0]
row["cvar95_normal_ewm"]                           # 0.0665 -> 6.7 % of the position in the worst 5 %

# what is my client's current portfolio risking?
qc.portfolio_risk({"NESN.SW": 500, "AAPL": 100}, cash=20_000, horizon="month")

# the research agent found something positive on Apple, with middling confidence
views = [{"symbol": "AAPL", "sentiment": 0.6, "confidence": 0.4}]
post = qc.expected_returns(max_assets=100, views=views)

# propose a portfolio for CHF 1 m that risks at most CHF 30 000 in the worst 5 % of weeks
res = qc.optimize(cash=1_000_000, horizon="week", confidence=0.95, cvar_limit=30_000,
                  views=views, risk_drift="zero", time_limit=300)

res["status"]                                      # 'Optimal'
res["cvar"]                                        # 30000.00  -- the limit binds exactly
for p in res["positions"]:
    print(p["symbol"], p["shares"], round(p["weight"], 4))

Every method passes its keyword arguments through to the API unchanged, so the field names in /docs are the field names here. A non-2xx answer raises QuantcoreError with the API's own message; an unreachable server raises it too, with unreachable = True.

8. From the command line

The same surface, for looking at things and for scripting:

quantcore-api universe --region Europe --sector "Health Care"
quantcore-api prices NESN.SW AAPL --freq weekly --tail 5
quantcore-api risk NESN.SW AAPL --horizon quarter --drift zero --space value
quantcore-api correlation NESN.SW NOVN.SW ROG.SW
quantcore-api risk-portfolio --holdings '{"NESN.SW": 500, "AAPL": 100}' --cash 20000
quantcore-api expected-returns --max-assets 50 --view AAPL:0.6:0.4
quantcore-api scenarios NESN.SW AAPL --model gbm --n-scenarios 4096
quantcore-api optimize --cash 1000000 --cvar-limit 30000 --horizon week --risk-drift zero
quantcore-api optimize --request my_request.json --json > result.json
quantcore-api update --wait                      # refresh prices and follow the job

--view SYMBOL:SENTIMENT[:CONFIDENCE[:STRENGTH]] is the shorthand for a sentiment view, repeatable. --json prints the raw answer (pipe it into jq, or save it as a fixture); it works before or after the subcommand. --request FILE takes a full JSON request, which is the right way to keep a complicated setup reproducible — its fields win over the flags.

9. The five things the project actually needs

Pick a universe. /data/symbols gives you rows and the counts per region, sector and country in one call, so a picker needs no second request. Pass the resulting symbols as symbols to /optimize to restrict the candidates; leave it out and the largest max_assets by market cap are used.

Measure what the client holds. /risk/portfolio with their share numbers and cash. Use method="historical" alongside the default "normal" and show both — where they disagree, the distribution is not Gaussian, and that is worth saying out loud.

Turn an assessment into a view. A view is a statement about an annual expected return with a degree of belief. Either explicit ({"assets": {"NESN.SW": 1.0}, "q": 0.09}: "Nestlé will return 9 %") or from sentiment ({"symbol": "NESN.SW", "sentiment": 0.6, "confidence": 0.4}). Sentiment in [−1, 1] scales strength (default 25 % p.a. at full sentiment); confidence in (0, 1] is the Idzorek confidence that sets Ω. Call /model/expected-returns before optimising to see what your view did to the posterior — that is the number you will have to defend.

Propose a new portfolio. /optimize with mode="new" and cash. Set cvar_limit (absolute CHF) or cvar_limit_fraction (of wealth). Constraints live in constraints, costs in costs:

{"mode": "new", "cash": 1000000, "horizon": "week", "confidence": 0.95, "cvar_limit": 30000,
 "risk_drift": "zero",
 "constraints": {"max_weight": 0.10, "max_sector_weight": 0.25, "max_positions": 25,
                 "min_position_value": 5000, "min_invested": 0.95},
 "costs": {"proportional_buy": 0.001, "proportional_sell": 0.001, "fixed_per_trade": 0}}

Rebalance an existing one. mode="rebalance" with holdings (shares) and the cash available. Transaction costs are charged and the result carries a before block, so you can show the client both states and what the trade bought them:

res = qc.optimize(mode="rebalance", holdings={"NESN.SW": 500, "AAPL": 100}, cash=50_000,
                  horizon="week", cvar_limit=30_000, risk_drift="zero",
                  constraints={"max_turnover": 200_000})
res["before"]["cvar"], res["cvar"]        # risk before and after
res["total_cost"], res["turnover"]        # what the change costs
[p for p in res["positions"] if p["trade"] != 0]   # the trades to place

10. Reading a result

Every optimisation answers with its own provenance. The fields you will quote:

field meaning
status Optimal | FeasibleFound (time limit — check the gap) | Infeasible | Error
solver cuopt (GPU) or highs (CPU) — which one actually produced this
reproducibility exact (HiGHS) | within_gap (cuOpt, 1e-4) | none
expected_pnl, expected_return over the horizon, in CHF and as a fraction
var, cvar, cvar_limit, confidence the risk figures and the limit they had to respect
positions per stock: shares, shares_before, trade, value, weight, cost, expected_return_ann
sector_exposure, invested, cash_end, total_cost, turnover the resulting portfolio
model covariance estimator, scenario model and seed, expected-return model, the views and the Ω that was calibrated from them
scenario_key, scenario_hash what the scenarios depend on, and a fingerprint of the draws
warnings read these — see below
timings seconds per stage

So a sentence in your advisory output can be backed end to end: "CVaR 95 % = CHF 29 999 over one week (mean–CVaR MILP, EWM λ = 0.99, GBM scenarios, seed 42, solved to optimality on HiGHS)."

warnings is not decoration. It is where the service tells you that the answer is technically correct and economically doubtful: a risk limit that could not bind, a covariance estimate too singular for the number of candidates, too few scenarios per candidate, an expected return above 60 % p.a. on a stock it decided to hold. Surface them or handle them; do not drop them.

Determinism. The same request gives the same scenarios (scenario_hash) and the same portfolio, on the same machine. Across machines — a different numpy or pandas — the scenario_key still matches while the digest may not, because floating-point results agree to 13–14 digits rather than bit for bit; the figures agree to well under a franc. Compare runs by scenario_key. For a demo that must be identical next week, freeze the data with a snapshot (POST /snapshots).

11. Things that will bite you

  • The service answers one request at a time while a solve runs. The MILP holds the interpreter. Do not build a UI that fires an optimisation and then polls something else — it will look hung.
  • time_limit is the budget of the whole request, not of each solver call (row generation and the Gaussian mode solve a sequence of MILPs). Without it, a large candidate set on HiGHS can run for hours. Set it. 300 s is a sane default.
  • The GPU solver is optional. solver="auto" uses cuOpt if it answers and HiGHS otherwise; the result says which, and adds a warning. A cuOpt outage degrades speed, never correctness.
  • risk_drift="model" (the default) measures the CVaR on scenarios that include the expected return. At long horizons the drift grows with h while the spread grows with √h, so a bullish forecast can push the whole loss tail above zero and the risk limit stops binding — the optimiser will then happily propose something absurd. Use risk_drift="zero" for anything beyond a few weeks; the objective still maximises expected return, but the risk measurement no longer believes the forecast.
  • Scenario count drives stability. The default is 16 per candidate (within 4 096…16 384). The reported figures are stable from ~2 000 on, but the composition moves between near-equivalent portfolios; about 64 per candidate pins it. The result warns below 8.
  • scenario_model="gaussian_exact" is fast but bounds the CVaR from below. It replaces the scenarios with linear cuts, and until those converge the limit can be overshot by a hair (a franc or two on a million, reported in warnings). Use it to explore; use "gbm" or "empirical" when the limit has to hold exactly.
  • Infeasible means your constraints contradict each other, most often min_invested against a tight cvar_limit, or a max_turnover that cannot reach the risk budget. Relax one and re-run.

12. Troubleshooting

symptom what it means
cannot reach the quant-core API at ... no service there. quantcore serve, or fix --url / QUANTCORE_API_URL
503 no market data yet the store is empty: POST /jobs/setup (~15 min) or mount a prepared data directory
422 with a field name your request does not match the schema; the message names the field. /docs has the valid values
/health shows cuopt.reachable: false the GPU solver is down; optimisations fall back to HiGHS and say so
FeasibleFound instead of Optimal the time limit hit; raise time_limit, or lower n_scenarios / max_assets
an optimisation "seems stuck" it is solving, and the service is single-threaded. timings afterwards shows where the time went
numbers differ slightly from a colleague's different environment, same inputs: compare scenario_key, not scenario_hash (§10)
Yahoo errors during a data job rate limiting; the client retries and opens a circuit. Re-run later, the stored data is intact

Everything failing at once? quantcore-api selftest tells you whether it is the service or you.

13. Where the details are

document what is in it
<url>/docs every endpoint and field, interactive — the contract
README.md the services, the library API, quick start
docs/ARCHITECTURE.md how the pieces fit, what depends on what, where the AI layer plugs in
docs/METHODOLOGY.md every formula, estimator and convention, with the measured numbers
docs/optimization_model.pdf the MILP derived from Rockafellar–Uryasev to whole shares, and the sampling theory
docs/measurement_and_models.pdf risk and performance measures, GBM and empirical innovations, Black–Litterman with Idzorek confidence