---
title: "Running Inference"
description: "REST API that triggers Argo Workflows for model inference on vxData resources."
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Running Inference

A simple REST API that triggers Argo Workflows for model inference on vxData resources. Results are written back to vxData automatically.

**API:** [`https://inference.fra.virdx.dev/`](https://inference.fra.virdx.dev/)

## How it works

```
User hits POST https://inference.fra.virdx.dev/<service>
->
Argo workflow is launched
->
Workflow image pulls specified resources from vxData, runs inference, uploads results back to vxData
->
User can retrieve results from vxData
```

## Repo structure

1. A FastAPI service that triggers Argo workflows for desired computations
2. A Docker image bundling all relevant repos (viseg, histo, vxdata-sdk, ...) for Argo-workflow-based inference jobs
3. A Python package (`virdx-inference`) for Python-based requests to the REST API

## Usage

**REST** — `POST /viseg`, `/histo-preprocessing`, `/diffsim` with a JSON body matching the schema below.

```bash
curl -X POST https://inference.fra.virdx.dev/histo-preprocessing \
  -H 'Content-Type: application/json' \
  -d '{
"input_resources": ["histo-scan/abc123"],
"tissue_detection_mag": 5.0,
"stats_level": 2,
"output_level": 0,
"crop_margin_frac": 0.05,
"tissue_detection_params": {
  "min_component_area": 200000,
  "fill_border_holes": false
},
"cpu": 8,
"memory": "32Gi",
"gpu": 0
  }'
```

**Python client:**

```bash
pixi add virdx-inference
```

```python
from inference.client import submit, VisegInference

job_id = submit(VisegInference(input_resources=["volume/abc123"]))
```

## Supported services

All services share these base fields from `InferenceRequest`:

| Parameter | Default | Description |
|---|---|---|
| `input_resources` | — | List of vxData resource identifiers (required) |
| `vxdata_api_url` | `"https://data.fra.virdx.dev/"` | vxData API endpoint |
| `cpu` | `4` | CPU cores allocated to the Argo workflow pod |
| `gpu` | `1` | GPUs allocated to the Argo workflow pod |
| `memory` | `"16Gi"` | Memory allocated to the Argo workflow pod |

From vxdata-sdk 2.x on, vxData requires a bearer token. The services build the
client without one, so the pod environment has to carry `VXDATA_TOKEN` — the SDK
reads it directly. Without it every vxData call returns 401. The API injects it
into the workflow pod via `podSpecPatch`, from the `vxdata-token-admin` secret
(key `VXDATA_TOKEN`) in the `argo-workflows` namespace. That secret is
provisioned out-of-band, not from a manifest in any repo.

Full parameter schemas: [`src/inference/schemas/__init__.py`](https://github.com/virdx/mono/blob/main/apps/inference/src/inference/schemas/__init__.py)

### `viseg`

Anatomy segmentation on `Volume` resources.

| Parameter | Default | Description |
|---|---|---|
| `model_id` | `"DEFAULT"` | Viseg model; `DEFAULT` resolves to the latest recommended model |
| `compute_uncertainty_score` | `false` | Run uncertainty estimation (logged only, not stored) |
| `batch_size` | `4` | Inference batch size passed to viseg |

### `histo-preprocessing`

Preprocessing pipeline for `HistoScan` resources.

| Parameter | Default | Description |
|---|---|---|
| `tissue_detection_mag` | `5.0` | Magnification level for tissue detection |
| `stats_level` | `2` | Pyramid level at which output statistics are computed |
| `output_level` | `0` | Pyramid level for image and label masks (`0` = native resolution) |
| `crop_margin_frac` | `0.05` | Fractional margin added around the detected tissue crop |
| `tissue_detection_params` | _(see below)_ | Fine-grained tissue detection parameters |

**`tissue_detection_params`** (`HistoTissueDetectionParams`):

| Parameter | Default | Description |
|---|---|---|
| `threshold` | `null` | |
| `blur_ksize` | `7` | |
| `early_dilate_ksize` | `51` | |
| `early_dilate_iters` | `5` | |
| `early_erode_ksize` | `null` | |
| `early_erode_iters` | `null` | |
| `pad` | `10` | |
| `min_raw_area` | `20000` | |
| `dilate_ksize` | `51` | |
| `dilate_iters` | `6` | |
| `erode_ksize` | `null` | |
| `erode_iters` | `null` | |
| `close_ksize` | `101` | |
| `fill_border_holes` | `true` | |
| `max_border_fraction` | `0.1` | |
| `min_component_area` | `100000` | |
| `min_area_frac_of_largest` | `0.0` | |

### `diffsim`

Diffsim simulations on `HistoMap` resources of `type == "DOMAIN_REPRESENTATION"`.

| Parameter | Default | Description |
|---|---|---|
| `protocol` | `"bpmri"` | Simulation protocol acting as a map to more complex protocol definition, we currently only allow dummy `bpmri` value |
| `walkers_count` | `10000000` | Number of random walkers |
| `voxel_resolution` | `1e-3` | MRI voxel resolution in metres (e.g. `1e-3` = 1 mm) |

## Planned services

- **`histo-domain-representation`** — schema stub exists; implementation pending v5 pipeline.
- **`ve2e`** — specify model and inference task, pass in patient(s). Will collect task-specific inference data, load model, perform inference, store predictions to vxData. Nice for benchmarking!

## Deployment

1. Build the API and inference images: `bash deployments/build.sh`
2. Trigger Argo CD to pick up the new image and re-deploy the API.
3. Any newly submitted inference request workflow will automatically use the latest `inference` image.

## Future ideas

- Endpoint for querying status and logs of jobs
- Endpoint for retrieving identifiers of produced resources
- Validation of input resources on API side (instead of only failing in the workflow)

Source: https://docs.virdx.dev/documentation/inference/index.mdx