What happens when a server dies

When a server stays unhealthy at your load balancer for 3 minutes, Hetscale starts a new server beside it from the node group’s current release, and the new server takes traffic once it passes the health check. The failed server leaves after that, or, in the cases below, never on its own.

When is a server treated as dead?

When your load balancer has reported it unhealthy for 3 counted minutes, while Hetzner reports neither a transitional status, such as a reboot, nor a lock on the server. A server that Hetzner reports as off counts as dead.

A minute counts only when Hetscale has a health reading for it. The clock does not run for a server that is already leaving or in maintenance, nor in a paused group or during a Rehearsal. A Live group acts, in Propose mode too, without waiting for approval; in Observe, the group’s timeline records that Hetscale would have started a server beside it.

What does Hetscale do then?

It starts one new server beside the failed one, from the group’s current release. The failed server stops counting as capacity and the new one takes its slot, so the group’s server count stays the same. The new server takes traffic only once your load balancer’s health check passes. The start is on the group’s timeline, and you get an alert.

The new server is held like any other create, by the cost guard, the euro ceiling and Hetzner’s stock, but never by your plan’s capacity. If your server type is sold out, it follows the fallback chain; when the chain is exhausted, it waits and is tried again as soon as stock returns.

A failed server that Hetscale created, in a group with a minimum of 2 or more, leaves once its replacement serves and the group’s “Keep a failed server” time has passed, counted from then. Its diagnostics are recorded first; then it goes through the drain window and is deleted. The setting is under Advanced in the group’s settings: “A server that fails its start, or that healing replaced, is kept this long for a look before it is deleted; 0 deletes it at once. Hetzner bills it meanwhile.”

Three kinds of failed server are never deleted: a server of a group with a minimum of 1, the seed, and every other server of yours in the group. Each stays in the group, not counted, until it is healthy again or you remove it, and Hetzner keeps billing it. The Servers page offers the delete for the servers Hetscale created; your own servers are yours to remove.

What happens with a minimum of one server?

The failed server is never deleted, and there is no other server to take the traffic. As of September 2026, a Hetzner load balancer whose only target is unhealthy answers 503, so requests fail until the new server passes its health check. A minimum of 2 keeps the group serving while a server is healed.

If the new server fails too, Hetscale stops: no second attempt for that failure, and an alert that says why the new server failed. Two failures in a row point at the release, not the server. This holds in every group, whatever its minimum. The failed server stays failed until it is healthy again, and a later failure can be healed again.

Is an overloaded server dead?

No. A server that is unhealthy while its own CPU is pinned, and that Hetzner reports as running, is overloaded, not dead; while its CPU is unknown, a scale-out rule of the group that holds right now stands in for it. The group scales out under its policy, your load balancer keeps the server out of rotation by itself, and the healing clock counts only the minutes without that condition. If the server is still unhealthy once the group could add capacity and an overload grace has passed, it is treated as failed and healed as above.

If no capacity can be added, nothing is replaced or removed. The group page shows “Attention required” with the reason: the group is at its maximum; the cost guard, the node cap, the euro ceiling or your plan’s capacity holds new servers; or Hetzner has no stock of the servers it needs. You get an alert, and again if it lasts. It ends when the server is healthy again, when it leaves, or when capacity can be added again, after which healing goes on; you get a notice either way.

What if the seed dies?

Healing never replaces the seed. Hetscale alerts you and starts a server beside it from the current release, and leaves the seed as it is: it is never deleted, and it is yours to look at and remove. The same holds for every other server of yours in the group. A seed you promoted to a managed server is healed like any server Hetscale created.

What if the server recovers first?

If the failed server is healthy again before its replacement serves, it counts again, and the replacement stays as ordinary capacity: the policy removes a server later under its usual scale-in rules. At the group’s maximum the replacement is released instead, so the group never runs more than its maximum. A kept server that recovers later counts again the same way, within the maximum. A healthy server is never deleted on a decision taken while it was failing.

What if every server fails at once?

When every server of a node group with a minimum of 2 or more fails the load balancer’s health check within 3 minutes of the first, while Hetzner shows them running, none of them is overloaded and the group’s load does not call for more servers, the cause is something they share: the load balancer, a service the health check depends on such as a database, or a Hetzner maintenance. A new server would meet the same fault, so Hetscale holds healing:

• It replaces none of them. Once the first of them has failed, you get one critical alert that says how many servers failed at once and names those likely causes. A blip of a minute or two on every server, such as a pipeline restarting them together or a short database failover, sends nothing.
• While the hold lasts, it creates no server but the test server below, so it neither scales out nor in. A server already leaving still leaves.
• New servers that fail their readiness during the hold do not count toward the pause after three failed new servers, so the group stays Live.
• After 10 minutes it starts one new server beside a failed one, as a test. If the test server passes the health check, the servers still unhealthy get their replacements at once. If it fails, healing stops for that server, with no second attempt, and the hold goes on.
• The hold ends when a server is healthy again, when a server turns out overloaded or the group’s load calls for more servers, or when no server is left behind the load balancer. When one of the old servers is healthy again, the others’ clocks start over, and a server still unhealthy gets its replacement 2 minutes after the hold ends. No scale-in follows for a whole scale-in window, because the load the servers reported while the load balancer sent them nothing is not your demand.

A group with a minimum of 1 heals each server as above. Servers whose CPU is pinned are overloaded, not failed (Is an overloaded server dead?). The timeline records the hold and its end; the end sends no mail.

What happens during a Hetzner maintenance?

Hetzner announces planned maintenance and lists running incidents on its status page, status.hetzner.com. Hetscale does not read that page, and Hetzner’s API has no maintenance status. What counts is what Hetzner shows for each server: minutes in which it shows the server starting, stopping, migrating or in another transitional status do not count toward the 3, and neither do minutes while an action locks the server.

A maintenance of your load balancer, or of the network in your location, can be different. If it makes every server of a group with a minimum of 2 or more fail the health check together while Hetzner shows them as running, Hetscale holds healing as above (What if every server fails at once?): it replaces none of them until its test server after 10 minutes, and the group is not paused. If only some servers fail, or the group’s minimum is 1, each failed server gets a new server beside it after 3 counted minutes, the seed included, though the seed itself stays and is never deleted. Each new server is held, like any other create, by the cost guard, the euro ceiling and Hetzner’s stock, and Hetzner bills it. A failed server that passes its health check again before its new server serves counts again, and the new one stays as ordinary capacity within the group’s maximum (What if the server recovers first?). A new server that does not pass the health check before the readiness deadline (15 minutes by default) fails as well, and healing stops for that failure. After three new servers in a row fail, the group is paused until a member chooses Resume.

So for an announced maintenance of your load balancer or of the network in your location, choose Hold for maintenance in Controls on the node group’s Overview, for 30 minutes, 1 hour, 4 hours or 24 hours. Until the hold ends, Hetscale scales in no server, replaces none that looks unhealthy, replaces no older server and moves no fallback server back to the group’s own type: an older or fallback server whose replacement already serves stays until the hold ends, and leaves after it only while that replacement still passes the health check. A server that really fails during the hold is replaced only after the hold ends. End maintenance, beside Controls, ends the hold early. Scale-out goes on during the hold, and every server it adds still has to pass the health check before the readiness deadline, so failures of those servers still count toward the pause. Pause stops scale-out too, and lasts until you choose Resume. Propose mode does not hold healing either, because healing acts without waiting for approval.

For work on one server, such as patching and restarting the seed, choose Hold for maintenance in the More menu of that server’s row on the Servers page. Until the hold ends, Hetscale neither removes that server nor replaces it.

What do I get afterwards?

An incident report for every healing, in every plan: the detection time, the decision and its reason, each step with its duration, the replacement’s Time to Capacity with its phases, the Hetzner action ids and the outcome: replaced, stopped or recovered. The group’s timeline links to it, and the panel lists your organisation’s reports, newest first. A capacity shortfall gets a report too; a scale-out does not.

Alerts reach your organisation’s members by mail. Alerts raised together arrive as one mail per organisation, whose subject gives the number of alerts and the worst level; the mail lists them group by group, each with its time and a line in words, and links to the group’s timeline. Times are shown in each member’s own time zone. On Pro and Team, the same alerts can also go to a Slack or Discord channel, one message per batch, or to a webhook of yours: Integrations.

On the first of the month, a summary mail adds up the month, healings included: Monthly summary.

What does this not cover?

• A broken release. Healing replaces a server, not the release it runs: two failures in a row stop healing, and three new servers in a row that fail their health check pause the group, except while healing is held because every server failed at once. The fix is a new release, or making an earlier one current: Releases, rollouts and rolling back.
• A release your own pipeline deployed to every server at once. If every server then fails its health check, healing holds as for a fault they share, and the fix is still a new release or making an earlier one current.
• A server or a group held for maintenance, from the server’s row on the Servers page or from the group’s Controls, for 30 minutes to 24 hours, is not healed while the hold lasts (What happens during a Hetzner maintenance?).
• How the failed server leaves: Removing a server without dropping requests.
• A sold-out server type: When your Hetzner server type is sold out.
• How long the new server takes to take traffic: How fast is a scale-out.