Inference Service

A simple REST API that triggers Argo Workflows for model inference on vxData resources. Results are written back to vxData automatically.

API: https://inference.fra.virdx.dev/

How it works

User hits POST https://inference.fra.virdx.dev/<service>
->
Argo workflow is launched
->
Workflow image pulls specified resources from vxData, runs inference, uploads results back to vxData
->
User can retrieve results from vxData

Repo structure

1. A FastAPI service that triggers Argo workflows for desired computations
2. A Docker image bundling all relevant repos (viseg, histo, vxdata-sdk, ...) for Argo-workflow-based inference jobs
3. A Python package (virdx-inference) for Python-based requests to the REST API

Usage

REST — POST /viseg, /histo-preprocessing, /diffsim with a JSON body matching the schema below.

curl -X POST https://inference.fra.virdx.dev/histo-preprocessing \
  -H 'Content-Type: application/json' \
  -d '{
    "input_resources": ["histo-scan/abc123"],
    "tissue_detection_mag": 5.0,
    "stats_level": 2,
    "output_level": 0,
    "crop_margin_frac": 0.05,
    "tissue_detection_params": {
      "min_component_area": 200000,
      "fill_border_holes": false
    },
    "cpu": 8,
    "memory": "32Gi",
    "gpu": 0
  }'

Python client:

pixi add virdx-inference

from inference.client import submit, VisegInference

job_id = submit(VisegInference(input_resources=["volume/abc123"]))

Supported services

All services share these base fields from InferenceRequest:

Parameter	Default	Description
input_resources	—	List of vxData resource identifiers (required)
vxdata_api_url	"https://data.fra.virdx.dev/"	vxData API endpoint
cpu	4	CPU cores allocated to the Argo workflow pod
gpu	1	GPUs allocated to the Argo workflow pod
memory	"16Gi"	Memory allocated to the Argo workflow pod

Full parameter schemas: src/inference/schemas/__init__.py

viseg

Anatomy segmentation on Volume resources.

Parameter	Default	Description
model_id	"DEFAULT"	Viseg model; DEFAULT resolves to the latest recommended model
compute_uncertainty_score	false	Run uncertainty estimation (logged only, not stored)
batch_size	4	Inference batch size passed to viseg

histo-preprocessing

Preprocessing pipeline for HistoScan resources.

Parameter	Default	Description
tissue_detection_mag	5.0	Magnification level for tissue detection
stats_level	2	Pyramid level at which output statistics are computed
output_level	0	Pyramid level for image and label masks (0 = native resolution)
crop_margin_frac	0.05	Fractional margin added around the detected tissue crop
tissue_detection_params	(see below)	Fine-grained tissue detection parameters

tissue_detection_params (HistoTissueDetectionParams):

Parameter	Default	Description
threshold	null
blur_ksize	7
early_dilate_ksize	51
early_dilate_iters	5
early_erode_ksize	null
early_erode_iters	null
pad	10
min_raw_area	20000
dilate_ksize	51
dilate_iters	6
erode_ksize	null
erode_iters	null
close_ksize	101
fill_border_holes	true
max_border_fraction	0.1
min_component_area	100000
min_area_frac_of_largest	0.0

diffsim

Diffsim simulations on HistoMap resources of type == "DOMAIN_REPRESENTATION".

Parameter	Default	Description
protocol	"bpmri"	Simulation protocol acting as a map to more complex protocol definition, we currently only allow dummy bpmri value
walkers_count	10000000	Number of random walkers
voxel_resolution	1e-3	MRI voxel resolution in metres (e.g. 1e-3 = 1 mm)

Planned services

• histo-domain-representation — schema stub exists; implementation pending v5 pipeline.
• ve2e — specify model and inference task, pass in patient(s). Will collect task-specific inference data, load model, perform inference, store predictions to vxData. Nice for benchmarking!

Deployment

1. Build the API and inference images: bash deployments/build.sh
2. Trigger Argo CD to pick up the new image and re-deploy the API.
3. Any newly submitted inference request workflow will automatically use the latest inference image.

Future ideas

• Endpoint for querying status and logs of jobs
• Endpoint for retrieving identifiers of produced resources
• Validation of input resources on API side (instead of only failing in the workflow)