Portfolio_Risk_Management/docs/SPARK_APPTAINER.md

quantcore on your DGX Spark (Apptainer, no root)

This guide sets up the quant service of the project on your own DGX Spark. The service covers market data, risk measurement, Black–Litterman expected returns and the mean-CVaR optimiser. Everything runs on the Spark itself:

• cuOpt, NVIDIA's GPU solver, runs in its own Apptainer container.
• quantcore, this repository, runs in a second Apptainer container. It calls cuOpt at http://127.0.0.1:5000.
• The API and the browser test bench listen on http://127.0.0.1:5001. You reach them from the Spark: its desktop browser, a terminal, a notebook, or the platform you build. They are not reachable from the network.

You need no root rights, no Docker and no VPN. One script, deploy/apptainer/qc.sh, does everything.

What you need

Requirement	Check
Apptainer 1.1 or newer	apptainer --version
The GPU	nvidia-smi -L lists the NVIDIA GB10. Without it, optimisation runs on HiGHS on the CPU.
About 5 GB in your home directory	The cuOpt image is 3.3 GB, quantcore 0.2 GB, the data 40 MB. Pulling cuOpt needs about 12 GB for a few minutes.
Internet access	Docker Hub for the images, PyPI for the Python libraries, Yahoo Finance for market data.

Quick start

Open a terminal on the Spark and run this one line:

wget -qO- https://gitea.fhgr.ch/herzogfloria/Portfolio_Risk_Management/raw/branch/main/install.sh | bash

If the Spark has no wget, one of these does the same:

curl -fsSL https://gitea.fhgr.ch/herzogfloria/Portfolio_Risk_Management/raw/branch/main/install.sh | bash
python3 -c "import urllib.request as u;print(u.urlopen('https://gitea.fhgr.ch/herzogfloria/Portfolio_Risk_Management/raw/branch/main/install.sh').read().decode())" | bash

The installer takes 20 to 30 minutes the first time and prints each step:

1. It checks Apptainer and the GPU. It also repairs Apptainer's temporary and cache folders if your account points them at a folder that does not exist.
2. It downloads the code to ~/Portfolio_Risk_Management. If the Spark has no git, it uses git from a small container.
3. It gets the cuOpt image and the quantcore image.
4. It builds the market data from Yahoo Finance, which takes about 15 minutes. The candidate stocks come from the list shipped in the repository, so every student gets the same universe. Companies that Yahoo no longer lists are left out, and setup names them in one summary line.
5. It starts the service, runs the self-test and installs the commands qc, quantcore and quantcore-api.

With a prepared data set from your lecturer, the data step takes seconds instead. Pass the file with --data:

wget -qO- https://gitea.fhgr.ch/herzogfloria/Portfolio_Risk_Management/raw/branch/main/install.sh | bash -s -- --data ~/Downloads/quantcore_data.tar.gz

Running the installer again is safe. It skips what is done and updates the rest.

When it is done, open http://127.0.0.1:5001/ui/ in the Spark's browser. The interactive API documentation is at http://127.0.0.1:5001/docs.

Working on the Spark over SSH from your laptop? Forward the port, then open the same address in the laptop's browser. Nothing else on the network can reach the service:

ssh -L 5001:127.0.0.1:5001 <your-user>@<your-spark>

The installer puts the commands qc, quantcore and quantcore-api into ~/.local/bin. Open a new terminal after the installation; a terminal that was already open does not find them. qc is the same as deploy/apptainer/qc.sh in the code folder. quantcore and quantcore-api are the commands of the README and of USAGE.md; on the Spark they run inside the container, so nothing needs to be installed with pip.

Daily use

Command	What it does
qc status	Shows what runs, whether cuOpt is reachable, and how much data there is. It warns when the API serves a different data set than the data folder.
qc logs	Shows the API log. qc logs cuopt shows the solver's log; add -f to follow.
qc start	Starts cuOpt and the API. Run it again after a reboot.
qc stop	Stops both containers.
qc restart	Stops and starts both.
qc update	Fetches the newest prices and FX rates from Yahoo Finance, then restarts the API.
qc upgrade	Downloads the newest version of the code, rebuilds and restarts.
~/Portfolio_Risk_Management/install.sh --rebuild-data	Sets the data aside in ~/quantcore/data.<date> and builds it anew (~10 min).

The containers keep running after you log out. They stop when the Spark reboots.

Using the service

In the browser. The test bench at http://127.0.0.1:5001/ui/ shows every service: the stock universe, risk and scenarios, expected returns and views, constraints, and the optimisation with a before/after comparison.

From your own code. This is how the platform you build uses the service. The API speaks JSON over HTTP, so any language works. Your code reaches it at http://127.0.0.1:5001. That address also works inside your own Apptainer containers, because Apptainer containers share the Spark's network. For Python there is a ready-made client:

from quantcore.client import QuantcoreClient

qc = QuantcoreClient("http://127.0.0.1:5001")
print(qc.health()["cuopt"]["reachable"])
risk = qc.asset_risk(["NESN.SW", "AAPL"], horizon="week")

Install the client into your platform's own virtual environment with pip install "git+https://gitea.fhgr.ch/herzogfloria/Portfolio_Risk_Management.git". Ubuntu refuses pip install outside a virtual environment. A quick script can also run with the container's Python, where quantcore is already installed: qc python my_script.py. USAGE.md describes every endpoint and the request fields.

From the command line, through the API. quantcore-api is the API client. It talks to your running service:

quantcore-api health
quantcore-api selftest                      # calls every endpoint, pass/fail table
quantcore-api risk NESN.SW AAPL --horizon week
quantcore-api optimize --cash 1000000 --cvar-limit 40000 --horizon week

From the command line, without the API. quantcore runs the services directly in the container. The API does not need to be running:

quantcore status
quantcore risk NESN.SW AAPL --horizon week
quantcore optimize --cash 1000000 --cvar-limit 40000 --horizon week --max-assets 50

quantcore serve starts the background service, like qc start. quantcore setup and quantcore update run through the service when it is running, so it picks up the new data. qc cli … and qc api … are the same commands under their qc names.

Settings

qc install writes the settings file ~/quantcore/quantcore.env. Edit it, then run qc restart. A value set in the shell wins over the file, for example QC_API_PORT=5011 qc start.

Setting	Default	Meaning
QC_API_PORT	5001	Port of the API and the test bench.
QC_CUOPT_PORT	5000	Port of the cuOpt server.
QC_CUOPT_START	auto	auto uses a cuOpt that already answers on the port, or starts one. yes always starts its own. no never starts one, so HiGHS solves every model.
QC_CUOPT_IMAGE	docker://nvidia/cuopt:26.8.0-cuda13.3-py3.14	Where the cuOpt image comes from. Set the path of a .sif file if your Spark already provides one.
QC_IMAGE	build	How quantcore gets its container; see the next section.
QUANTCORE_*		Passed into the container unchanged. Examples are QUANTCORE_CUOPT_TIME_LIMIT and QUANTCORE_ENABLE_UI. All of them are listed in quantcore/config.py.

Several users on one Spark. Ports belong to the whole machine. If someone else already uses port 5001, qc start stops and suggests a free port to put into your settings file. A cuOpt that another user already runs on port 5000 is simply used: cuOpt keeps no state between requests.

How quantcore gets its container

QC_IMAGE=build is the default. qc install then builds the image quantcore.sif from this repository with Apptainer's unprivileged build. Some Spark installations do not allow unprivileged builds. In that case qc install falls back by itself to a setup that needs no build. It pulls a plain Python image and installs quantcore into a virtual environment in ~/quantcore/venv. The libraries and the results are the same, because both setups install the pinned versions in deploy/apptainer/constraints.txt. You can choose the fallback directly with QC_IMAGE=venv.

QC_IMAGE also accepts the path of a prebuilt quantcore.sif, or a docker:// or oras:// address to pull one from.

New versions of the code come with qc upgrade. It downloads the changes, rebuilds quantcore and restarts the service.

How it works

 DGX Spark, your user account, no root
 ┌────────────────────────────────────────────────────────────────────────────┐
 │  browser / your platform / qc api                                          │
 │        │ HTTP                                                              │
 │        ▼                                                                   │
 │  127.0.0.1:5001   instance "quantcore-api"    quantcore.sif                │
 │                   FastAPI + test bench        ~/quantcore/data -> /data    │
 │        │ HTTP (msgpack)                                                    │
 │        ▼                                                                   │
 │  127.0.0.1:5000   instance "quantcore-cuopt"  cuopt.sif, started with --nv  │
 │                   cuOpt server                ──► GB10 GPU                 │
 └────────────────────────────────────────────────────────────────────────────┘

• An Apptainer instance is a container that runs in the background, like a service. apptainer instance list shows yours.
• Apptainer containers use the Spark's network directly, with no port mapping. A server inside a container that listens on 127.0.0.1 is therefore reachable from the Spark, and only from it.
• The images are read-only. Everything that changes lives in ~/quantcore/data, which is mounted into the container as /data.
• The containers start with --cleanenv. Variables from your shell, such as an activated virtual environment or PYTHONPATH, cannot interfere. qc.sh passes only the settings above, plus proxy variables if you have them.
• NVIDIA's cuOpt image sets its home directory to /opt/cuopt. That folder is writable under Docker but read-only in an Apptainer image, and the solver would crash on its first job when CuPy writes its kernel cache there. qc.sh therefore starts the cuOpt server with its home directory in ~/quantcore/cuopt-home.
• When cuOpt is not reachable, or stops during a solve, every optimisation falls back to HiGHS on the CPU. The result says which solver produced it.

Path	Content
~/quantcore/quantcore.env	your settings
~/quantcore/data/	universe, prices, FX, snapshots
~/quantcore/images/cuopt.sif	the cuOpt image
~/quantcore/images/quantcore.sif	the quantcore image; in the fallback, python.sif plus ~/quantcore/venv/
~/quantcore/cuopt-home/	cuOpt's caches (CuPy kernels, CUDA)
~/.apptainer/instances/logs/	instance logs, also shown by qc logs
~/.apptainer/cache/	downloaded image layers; qc clean-cache frees the space

Troubleshooting

Symptom	Cause and fix
port 5001 is taken	Another user or program uses the port. Set the suggested QC_API_PORT in ~/quantcore/quantcore.env, then run qc start.
qc status shows cuOpt as not answering	Run qc logs cuopt. Check that nvidia-smi works. Then qc restart. Optimisations keep working on HiGHS in the meantime.
The cuOpt log says Read-only file system: '/opt/cuopt/.cupy'	cuOpt was started without qc.sh. Start it with qc start, or give it a writable home directory as described in "Running cuOpt by hand".
The test bench or /optimize answers 503, "no market data yet"	The data folder is empty. Run qc status: it lists earlier data sets in ~/quantcore/data.<date> and the command to restore one, qc data-import <folder>. Otherwise build the data with qc setup --offline (~10 min).
qc status warns that the API serves a different data set	Run qc restart.
Yahoo errors during qc setup or qc update	Yahoo limits how fast it answers. Run the command again later; stored data stays intact.
FATAL: ... failed to create build parent dir: stat /scratch/...	Your account sets APPTAINER_TMPDIR to a folder that does not exist. The installer and qc now detect this, use a folder that works, and save the choice in ~/quantcore/quantcore.env. Run the installer again.
git: command not found	Use the one-line installer. It needs no git.
Hundreds of red Quote not found for symbol lines during setup	Your installation predates the fix. Its setup used guessed symbols, so many European companies are missing from your data. Run qc upgrade, then ~/Portfolio_Risk_Management/install.sh --rebuild-data.
qc: command not found or quantcore: command not found right after the installation	Open a new terminal, or run export PATH="$HOME/.local/bin:$PATH".
error: externally-managed-environment from pip install	That is the laptop route of the README. On the Spark nothing needs to be installed with pip: quantcore and quantcore-api already work. For your own code, use a virtual environment.
qc install stops with "no 'instance run'"	The Spark's Apptainer is older than 1.1. Ask the administrator to update it.
The cuOpt pull fails with "no space left on device"	Apptainer unpacks images in /tmp. Point it at a larger directory: APPTAINER_TMPDIR=~/quantcore/tmp qc pull-cuopt.
Model status FeasibleFound instead of Optimal	The time limit was reached. Raise time_limit in the request or QUANTCORE_CUOPT_TIME_LIMIT, or reduce n_scenarios or max_assets.

Running cuOpt by hand

qc start does this for you. If you want to run the cuOpt image yourself, for example for another project, start it like this:

apptainer pull cuopt.sif docker://nvidia/cuopt:26.8.0-cuda13.3-py3.14
mkdir -p ~/cuopt-home
apptainer instance run --nv --cleanenv --bind ~/cuopt-home cuopt.sif my-cuopt \
    env HOME=$HOME/cuopt-home python -m cuopt_server.cuopt_service -i 127.0.0.1 -p 5000
curl http://127.0.0.1:5000/cuopt/health
apptainer instance stop my-cuopt

For the lecturer

Data set for the students. Many students running qc setup at the same time from one network can hit Yahoo's rate limits. A prepared data set avoids that, and every student then works on the same data. Pack it on a machine that has the data:

qc data-export quantcore_data.tar.gz              # or, from any data directory:
tar -czf quantcore_data.tar.gz -C <data dir> --exclude ./jobs --exclude '*.log' .

The archive is about 30 MB. It is not part of the public repository: Yahoo's terms do not allow redistributing their data.

Prebuilt image. Build quantcore.sif on any aarch64 Linux machine where you can build images, then hand out the file:

apptainer build quantcore.sif deploy/apptainer/quantcore.def    # from the repository root

Students set QC_IMAGE=/path/to/quantcore.sif in their settings file. A copy built and tested on the lecturer's Spark is described in the verification note below.

The lecturer's own Spark runs the Docker setup in SPARK_DEPLOYMENT.md. That setup needs root and Docker. Students do not need it.

Verification. The setup was tested end to end on 7 October 2026 on a DGX Spark: NVIDIA GB10, driver 580.159, Ubuntu 24.04, Apptainer 1.5.4, as a user without root. That Spark allowed no unprivileged image builds, so qc install took the virtualenv fallback by itself. The image mode was then tested with a prebuilt quantcore.sif. In both modes:

• qc api selftest --full passed on every endpoint, and the optimisation ran on cuOpt on the GPU.
• qc update fetched the newest Yahoo prices through the running API.
• A taken API port stopped qc start with a suggested free port, and that port worked.
• The test suite passed inside the container, including the round trip to the live cuOpt server.

The one-line installer was then tested on a stripped-down machine. It had no git, curl, wget or python3, a home folder named like a domain account (/home/name@edu.local), and APPTAINER_TMPDIR pointing to a /scratch folder that does not exist. The installer repaired the folders, fetched the code with git from a container, installed, started and passed the self-test. qc upgrade picked up a new commit, rebuilt quantcore and restarted it in under a minute. qc setup built a data set from scratch inside the container.

Where the candidate stocks come from. The installer runs qc setup --offline. It takes the candidates from the lists shipped in quantcore/data/static/: the S&P 500 and the iShares STOXX Europe 600 holdings of September 2026, from which the course data set was built. qc setup without --offline fetches the current lists instead. In October 2026 iShares moved the German download, so setup now reads the same file from iShares' Swiss site. If that fails too, setup uses the shipped list. It no longer uses Wikipedia's STOXX 600 table, whose codes mostly do not exist on Yahoo, for example BNPP.PA instead of BNP.PA.