Pyo
Pyo Cloud
AIby PyoOfficial template

Deploy vLLM on your own servers

Serve an open model from Hugging Face with an OpenAI-compatible API, on NVIDIA GPUs.

Get started vLLM website

What Pyo sets up

Image
docker.io/vllm/vllm-openai:v0.33.0
Reached
Inside the cluster only
Runs as
One replica with its own data
Replicas
1
Requests
2 vCPU · 8 GiB
Limits
8 vCPU · 32 GiB
Storage
100 GiB at /models
GPU
1 NVIDIA GPU, required
Health check
HTTP /health

What you can configure

  • ModelQwen/Qwen3-0.6B

    A Hugging Face model id. It has to fit in the GPU's memory.

  • Hugging Face tokenGenerated, or your own

    Only for gated models (e.g. Llama).

How to deploy it

  1. 1

    Install Pyo

    Run the setup command on your server.

  2. 2

    Choose vLLM

    Open Templates in the dashboard and pick it. Pyo fills in sensible defaults.

  3. 3

    Deploy

    Pick a server and deploy. Pyo runs it and follows the rollout.

Ports

  • api 8000/HTTPPrivate

Good to know

  • Needs an NVIDIA GPU server in the cluster.
  • The first start downloads the model to the volume: it can take many minutes.
  • Talk to it at http://<service>:8000/v1, like the OpenAI API.

Related templates

All templates

Get Pyo running on your server

Run Pyo on any machine, from your laptop or VPS to a dedicated server or homelab.

curl -fsSL https://pyo.dev/install.sh | bash
View on GitHub