Scaling a service

ReplicasCopy Anchor Link

Set how many replicasOne running copy of a service. More replicas share the traffic and keep it up if one stops. of a service run, from the dashboard or through MCP. Scaling takes effect without a redeploy.

There's no fixed maximum: run as many replicas as your servers have room for. If you ask for more than fits, Pyo still accepts it and tells you how much CPU and memory they need against what's free. The replicas that don't fit show as Waiting for room and start once there's space, for example after you add a server.

A very large change asks you to confirm first: one to 20 or more replicas that's over 10 times what the service had, or over twice what your servers have room for.

AutoscalingCopy Anchor Link

Pyo can choose the number of replicas for you, between a minimum and a maximum, based on CPU use and, optionally, memory use. Open a service, choose Scale, turn on Autoscale and set:

• Minimum: the fewest replicas it runs, even when idle.
• Maximum: the most it can run. Pyo tells you if your servers don't have room for that many.
• CPU target: the average use to aim for, as a percentage of each replica's CPU requestThe CPU Pyo reserves for each replica. You set it under Resources, below..
• Memory target: optional. Turn on Also scale on memory to aim for a memory use too, as a percentage of each replica's memory request. When both are on, whichever needs more replicas wins.

With a target of 70% and a request of 100mCPU is measured in thousandths of a core, so 100m is a tenth of one core., Pyo adds replicas once they average more than 70m each. It removes them a few minutes after use drops, so a short dip doesn't cut capacity.

Deploys keep the current number of replicas, so a deploy during a busy period won't shrink your service. If you turn autoscaling off, the service stays at the size it had reached.

Autoscaling has some limits:

• It follows CPU, and memory if you turn it on, but not requests per second.
• Many programs don't give memory back once they've used it. With a memory target, such a service may scale up and not come back down, so set it with care.
• Set realistic requests, because use is measured against them. If the request is much lower than what the service needs, it scales up early.
• It can only add replicas as far as your servers have room. If they're full, new replicas wait until there's space. Pyo doesn't add servers.
• Services with a volume run as one replica, so they can't autoscale.
• It needs metrics-serverA small system service that reports how much CPU and memory each replica uses., which comes with k3s. Keep it enabled.

Deploys without downtimeCopy Anchor Link

Pyo starts a new replica and waits for it to be ready before stopping an old one, so deploys don't drop traffic. A stopping replica keeps serving briefly while traffic moves away from it.

Services with a single-writer volumeA volume that only one copy of a service, on one server, can use at a time. are the exception: they restart on deploy, to protect their data.

ResourcesCopy Anchor Link

Each service has its own CPU and memory requests and limits. The request is what Pyo reserves for it, and the limit is the most it can use.

Choosing where services runCopy Anchor Link

By default, a service can run anywhere in its cluster. You can narrow that down by:

• Region: servers in one region.
• Tags: servers with all of the given tags, like gpu.
• Server: specific servers by name.

A service pinned to one server can't move if that server goes down.

Replicas also spread across regions on their own. See Serving traffic by region.

Self-healingCopy Anchor Link

Crashed containers restart on their own. With more than one server, stateless services move to a healthy server if one goes down.