Configure endpoint autoscaling
Endpoint scaling is controlled by the sizing configuration sent when an endpoint is deployed. In the current Workbench UI, endpoint presets populate the CPU, memory, GPU count, compute pool, configurations, and replica bounds used by the deployment.
Review and apply endpoint replica bounds through inference presets.
Review preset sizing
Open inference presets
Click Workbench in the left sidebar, then click Compute > Presets and select the Inference tab.
Open a preset
Click the preset you plan to use for the endpoint.
Check replica bounds
Review the preset's CPU, memory, GPU count, compute pool, configurations, minimum replicas, and maximum replicas.
Apply autoscaling during deployment
Deploy an endpoint
Start the Deploy endpoint workflow.
Select a preset
On the Tuning step, select the inference preset that has the replica bounds you want. The wizard uses the preset to populate the endpoint sizing payload.
Confirm the deployed endpoint
After deployment, open the endpoint detail page. The Related components panel shows the autoscaler when min or max replicas are configured, and the Status panel shows ready, available, and target replica counts.
Success looks like this: the endpoint detail page shows the expected target replica count and autoscaler-related component information.
Scale-to-zero requires a preset with a minimum replica count of 0. Expect cold-start latency when traffic resumes after the endpoint has scaled down.