Running Inference

A simple REST API that triggers Argo Workflows for model inference on vxData resources. Results are written back to vxData automatically.

API: https://inference.fra.virdx.dev/

How it works

User hits POST https://inference.fra.virdx.dev/<service>
->
Argo workflow is launched
->
Workflow image pulls specified resources from vxData, runs inference, uploads results back to vxData
->
User can retrieve results from vxData

Repo structure

A FastAPI service that triggers Argo workflows for desired computations
A Docker image bundling all relevant repos (viseg, histo, vxdata-sdk, …) for Argo-workflow-based inference jobs
A Python package (virdx-inference) for Python-based requests to the REST API

Usage

REST — POST /viseg, /histo-preprocessing, /diffsim with a JSON body matching the schema below.

curl -X POST https://inference.fra.virdx.dev/histo-preprocessing \
  -H 'Content-Type: application/json' \
  -d '{
    "input_resources": ["histo-scan/abc123"],
    "tissue_detection_mag": 5.0,
    "stats_level": 2,
    "output_level": 0,
    "crop_margin_frac": 0.05,
    "tissue_detection_params": {
      "min_component_area": 200000,
      "fill_border_holes": false
    },
    "cpu": 8,
    "memory": "32Gi",
    "gpu": 0
  }'

Python client:

pixi add virdx-inference

from inference.client import submit, VisegInference

job_id = submit(VisegInference(input_resources=["volume/abc123"]))

Supported services

All services share these base fields from InferenceRequest:

Parameter	Default	Description
`input_resources`	—	List of vxData resource identifiers (required)
`vxdata_api_url`	`"https://data.fra.virdx.dev/"`	vxData API endpoint
`cpu`	`4`	CPU cores allocated to the Argo workflow pod
`gpu`	`1`	GPUs allocated to the Argo workflow pod
`memory`	`"16Gi"`	Memory allocated to the Argo workflow pod

From vxdata-sdk 2.x on, vxData requires a bearer token. The services build the client without one, so the pod environment has to carry VXDATA_TOKEN — the SDK reads it directly. Without it every vxData call returns 401. The API injects it into the workflow pod via podSpecPatch, from the vxdata-token-admin secret (key VXDATA_TOKEN) in the argo-workflows namespace. That secret is provisioned out-of-band, not from a manifest in any repo.

Full parameter schemas: src/inference/schemas/__init__.py

`viseg`

Anatomy segmentation on Volume resources.

Parameter	Default	Description
`model_id`	`"DEFAULT"`	Viseg model; `DEFAULT` resolves to the latest recommended model
`compute_uncertainty_score`	`false`	Run uncertainty estimation (logged only, not stored)
`batch_size`	`4`	Inference batch size passed to viseg

`histo-preprocessing`

Preprocessing pipeline for HistoScan resources.

Parameter	Default	Description
`tissue_detection_mag`	`5.0`	Magnification level for tissue detection
`stats_level`	`2`	Pyramid level at which output statistics are computed
`output_level`	`0`	Pyramid level for image and label masks (`0` = native resolution)
`crop_margin_frac`	`0.05`	Fractional margin added around the detected tissue crop
`tissue_detection_params`	(see below)	Fine-grained tissue detection parameters

tissue_detection_params (HistoTissueDetectionParams):

Parameter	Default	Description
`threshold`	`null`
`blur_ksize`	`7`
`early_dilate_ksize`	`51`
`early_dilate_iters`	`5`
`early_erode_ksize`	`null`
`early_erode_iters`	`null`
`pad`	`10`
`min_raw_area`	`20000`
`dilate_ksize`	`51`
`dilate_iters`	`6`
`erode_ksize`	`null`
`erode_iters`	`null`
`close_ksize`	`101`
`fill_border_holes`	`true`
`max_border_fraction`	`0.1`
`min_component_area`	`100000`
`min_area_frac_of_largest`	`0.0`

`diffsim`

Diffsim simulations on HistoMap resources of type == "DOMAIN_REPRESENTATION".

Parameter	Default	Description
`protocol`	`"bpmri"`	Simulation protocol acting as a map to more complex protocol definition, we currently only allow dummy `bpmri` value
`walkers_count`	`10000000`	Number of random walkers
`voxel_resolution`	`1e-3`	MRI voxel resolution in metres (e.g. `1e-3` = 1 mm)

Planned services

histo-domain-representation — schema stub exists; implementation pending v5 pipeline.
ve2e — specify model and inference task, pass in patient(s). Will collect task-specific inference data, load model, perform inference, store predictions to vxData. Nice for benchmarking!

Deployment

Build the API and inference images: bash deployments/build.sh
Trigger Argo CD to pick up the new image and re-deploy the API.
Any newly submitted inference request workflow will automatically use the latest inference image.

Future ideas

Endpoint for querying status and logs of jobs
Endpoint for retrieving identifiers of produced resources
Validation of input resources on API side (instead of only failing in the workflow)