Deploy an endpoint
Deploy an endpoint when you want to serve a model behind an authenticated HTTP URL. Workbench supports LLM and predictive endpoints, multiple model sources, and runtime tuning through inference presets.
Deploy a model as a Workbench inference endpoint using the three-step endpoint wizard.
Open KServe endpoints
Click Workbench in the left sidebar, then click Serve > KServe > Inferences.
Start the endpoint wizard
Click Deploy endpoint. The Deploy Inference Endpoint modal opens.
Complete Identity
Enter an Endpoint name and optional Description.
Complete Model source
Select a source type and fill in the matching fields.
| Source type | What you provide |
|---|---|
| S3 | Bucket and optional prefix. |
| HuggingFace | Repository, optional revision, and optional HuggingFace secret for gated models. |
| PVC | Claim name and optional path. |
| Model Registry | Registered model and version. |
| Custom URI | Container or model URI. |
For non-HuggingFace sources, also choose the endpoint Kind: LLM or Predictive. HuggingFace sources use runtime suggestions to infer the kind when possible.
Complete Tuning
Select an inference Preset, Runtime, and Framework. Presets fill in sizing values such as CPU, memory, GPU count, compute pool, configuration defaults, and replica bounds.
Deploy
Click Deploy endpoint.
Open endpoint details
The endpoint appears in the Inferences list. Click the row to open the detail page, view status, copy the request URL, and inspect related components.
Success looks like this: the endpoint reaches Running, at least one predictor replica is ready, and requests to the displayed URL return responses.
LLMs can report an active Kubernetes service before weights are fully loaded into GPU memory. If requests time out immediately after deployment, check the endpoint detail page and logs before changing the configuration.