A new server never becomes ready

Last updated 2026-10-20

A new server becomes ready only when your load balancer’s health check passes for it. Until then it gets no traffic. If the check never passes, Hetscale removes the server and reports it. Three such servers in a row pause the node group. The seed passes the same check, so the cause is usually something that is different on the copy. The copy boots a snapshot of the seed’s disk, gets its own addresses and runs its own first boot. The first three headings below are the panel’s words for what you see. The questions after them are the usual causes, each with its fix. A server that fails after it served is a matter for healing (What happens when a server dies). A server that Hetzner has no stock for is covered on A node group is servers short. A server refused because of an account limit is covered on A node group is paused.

Health failed

A new server of a Live node group did not pass the load balancer’s health check before the group’s Readiness deadline. The deadline counts from the moment Hetzner finished creating the server. It is 15 minutes by default. Before Live, Readiness deadline on the group’s Go Live page sets it from 5 to 45 minutes, from release v1 on and not while a Rehearsal runs; once the group is Live, Advanced in its Settings holds it, from 5 to 60 minutes. The timeline records “Server failed” with the phase the server stopped in, such as “Health checking → Health failed”, and the reason “Not ready before the deadline”. The alert reads “‹server› never became ready and is removed.” The server never got traffic.

Hetscale deletes the server once the group’s Keep a failed server time has passed (0 minutes by default). The timeline then shows “Shutting down” with the reason “Health check failed”. While the server is kept, the Failed tab of the Servers page shows it as “Health failed”. If the group still needs the capacity, it starts another server.

Two other cases end the same way:

With the webhook adapter, a failed report from your pipeline stops the server at once, as “Deploy failed”.
If Hetzner refuses to add a new server to the load balancer, the server fails as “Health failed” at once and the group pauses. The timeline names Hetzner’s answer (A node group is paused).

The same rule holds for a replacement that healing starts and for a new server that replaces an older one. If that server fails, healing stops for it, and replacing older servers stops (Releases, rollouts and rolling back).

Rehearsal failed

The Rehearsal’s server did not become ready, and the Rehearsal removed it. You see this in three places:

The timeline records “Rehearsal failed” with the reason in brackets.
The Rehearsal step on the Go Live page reads “Failed: ‹reason›. Start it again.”
The Rehearsal section of that page shows when the run failed and why.

The Rehearsal’s server gets the readiness deadline set on the Go Live page, 15 minutes unless you changed it (Health failed). The reasons you can see:

“the load balancer health check did not pass”: the server reached the readiness deadline without passing the check. The causes below apply.
“the application did not start”: your pipeline reported failed for the server. This happens with the webhook adapter only.
“the deploy pipeline never reported the server ready”: with the webhook adapter, your pipeline did not report the server ready before the readiness deadline, so it never joined the load balancer. That includes the case where Hetscale could not deliver the event to your pipeline: a failed delivery is retried for about 38 minutes, longer than the default deadline (Did your pipeline report the server ready in time?).
“the server was not ready within an hour”: the load balancer did not route to the server within an hour of the start.
“Hetzner could not create the server”: the create itself failed. The timeline names Hetzner’s answer.

Fix the cause, then choose Start rehearsal again. Hetzner bills each run’s server for its started hours. Each start counts failed new servers from zero again, so failed Rehearsals never pause the node group.

Some fixes are outside the seed’s disk: the health check, a Cloud Firewall’s rules, the network’s route, an allowlist. Those apply to the next new server without a new release.

A fix on the seed’s disk reaches new servers only through a new snapshot. Before Live, while no Rehearsal has passed, fix the seed and choose Take release v1 again on the Go Live page: it runs the same preflight and replaces release v1 with a new snapshot of the seed, and the next Rehearsal boots from it. Only the image changes; the server type, the networks, the firewalls and the public addresses stay those of the first take. The earlier snapshot is no longer used, and once the group is Live, Delete unused snapshots on its Deployment page removes it. Once the node group is Live, Take a new release on its Deployment page takes a new snapshot. With the webhook adapter, new servers always boot the snapshot taken as release v1, so your pipeline has to add what that disk lacks when it deploys.

New servers never became ready

Three new servers in a row failed at the deploy or at the readiness deadline, so Hetscale paused the node group instead of creating more. The State card on the group’s Overview shows this reason, and the mail reads “The group was paused: several new servers in a row never became ready.”

The third server is not deleted while the group is paused. It stays on the Failed tab of the Servers page, with its addresses under Details, and Hetzner bills it. Hetscale deletes it after you resume. The count starts again when a new server becomes ready and when you resume. Find the cause below, fix it, then choose Resume (A node group is paused).

Where do I see why a server failed?

The timeline. The “Server failed” entry names the phase the server stopped in. “Health checking → Health failed” means the server was a target of the load balancer and never passed the check. “Deploying → Health failed” means it never joined the load balancer. With the webhook adapter, that means your pipeline never reported it ready.
The Health page. The group’s Health page lists every change in how the load balancer saw each server. Each row has Hetzner’s status of the server and, where Hetzner gives them, its detail and the HTTP status the check got. If nothing answers, look at the start, the address or a firewall. An unexpected status or body points at the application, or at the check’s path or domain.
The server itself. On a Live node group, set Keep a failed server under Advanced in Settings, up to 120 minutes, before the next try. After a pause, the third failed server is still there anyway. A kept server gets no traffic, and Hetzner bills it meanwhile. Its addresses are under Details on the Servers page. Log in the way you log in to the seed and read the application’s and the system’s logs. Hetscale does not log in to your servers. Pick at least one key under SSH keys first. Without one, Hetzner gives each new server an expired root password and mails it to your Hetzner account, and a login as root first asks for that password and a new one (quickstart, step 3).
After a Rehearsal. A Rehearsal removes its server when it fails, so only the timeline and the Health page are left. The Rehearsal section of the Go Live page shows when the run failed and why, but not the phases.

Does the application start on boot?

A copy starts from the disk, not from the running seed. Anything you started by hand on the seed is not running on the copy. Give every container a restart policy (restart: unless-stopped or always in Compose), and enable every service the application needs (systemctl is-enabled <service> answers enabled).

A container with unless-stopped that was stopped when the release was taken stays stopped on every copy. To list each container’s policy, run this on the seed: docker inspect --format '{{.Name}} {{.HostConfig.RestartPolicy.Name}}' $(docker ps -aq).

This is the “Starts on boot” line you confirmed at the first release (Go Live). Fix it on the seed, then take a new release.

Does the application answer on the address the load balancer checks?

If the seed has a private network, Hetscale adds a new server to the load balancer by its private IP. Otherwise it uses the public IP. This is true even if you added the seed itself by its public IP.

A copy also has addresses of its own. An application, a proxy or a published Docker port bound to one of the seed’s addresses answers nowhere on the copy. An example is <seed-ip>:80:80 in Compose. Bind to all interfaces (0.0.0.0) and let the firewall decide who reaches the port, then take a new release.

To test it, call the endpoint from another server in the network, the way the load balancer does: step 2 of the health endpoint guide.

Does the seed pass the same health check?

Every server of the node group gets the same health check from the load balancer’s service: the same protocol, port, path, domain and accepted status codes. If the load balancer has more than one service, such as one for HTTP and one for HTTPS, a server is ready only when the check of every service passes.

If the seed is unhealthy too, the check is wrong, not the copy. You can see the seed’s state in the Hetzner Console under the load balancer’s targets, or on the group’s Health page.

If your proxy or framework routes by host name or checks it, set your domain as the check’s domain. Without it, the check lands on another site or on an error. Coolify’s proxy behaves this way (Coolify). A TCP check proves only that the port is open. The settings are in step 3 of the health endpoint guide.

Does a firewall on the server let the check in?

Hetzner Cloud Firewalls filter only a server’s public interface. New servers get the Cloud Firewalls the seed had when release v1 was taken. A server without public addresses gets none. A change to the rules of one of those firewalls reaches every server that has it.

A firewall inside the operating system is copied with the disk. Suppose it admits the load balancer only on the public interface, or only from the load balancer’s public address. Then it refuses the check on a copy that the load balancer reaches through its private IP. Admit the health check’s port from the private network’s range, then take a new release.

Does the health check depend on a service that knows only the seed?

A copy has its own private address. If the seed has public addresses, the copy also gets new public ones. A database, a cache, a secrets store or a partner’s API that admits only the seed’s addresses refuses the copy. If the health endpoint calls that service, the check fails. If it does not, the copy serves errors once it gets traffic.

If the service sits in the same Hetzner network, admit the private network’s range. Otherwise, give the node group one outbound address: Allowlists and new servers. Keep the health endpoint off shared services: Write a health endpoint your load balancer can trust.

Can a server without public addresses reach the internet?

If the seed has no public IPv4, new servers have none either. A copy with no public address at all reaches the internet only through your NAT server. That needs two things:

a route 0.0.0.0/0 in the network;
a default route in the copy’s own network files that sorts before cloud-init’s.

A route you set by hand on the seed does not reach the copies. With IPv6 only, a registry, mirror or API that has no IPv6 address is out of reach. The preflight at Take release v1 warns under “Route to the internet”.

If the application needs the internet to start, the copy fails its check. If it needs the internet only later, the copy passes the check and then fails its requests. The setup is in Servers without public addresses.

Is everything the application needs in the snapshot?

A new server boots a snapshot of the seed’s disk as it was when the snapshot was taken, .env included. With the snapshot adapter, that is the current release’s snapshot. With the webhook adapter, it is release v1’s. The snapshot does not hold:

files on a Hetzner Volume attached to the seed. A volume is not part of the snapshot, and new servers get none. A volume shows as its own mount in df -h;
anything that changed on the seed after the snapshot was taken, including a new .env value;
what the seed fetches at start from a service that admits only its address (above).

Put the files the application needs to start on the server’s own disk, and keep state in a service every copy reaches. Then take a new release. With the webhook adapter, your pipeline can add what the snapshot lacks when it deploys.

Does the first boot pull images or packages?

With both adapters, a new server boots from the seed’s disk, so the images and packages on it are already there. Some starts fetch anyway: a start script that pulls images first, a tag the disk does not hold, or an upgrade run at boot. Such a start depends on a registry or mirror that answers in time and does not refuse or rate-limit a new address.

Start from what is on the disk instead. Pin images the disk holds and keep upgrades out of the boot, then take a new release.

With the webhook adapter, your pipeline pulls the release. If the pull fails, it shows as “Deploy failed” when the pipeline reports failed, or as a server that never reported ready.

Did your pipeline report the server ready in time?

With the webhook adapter, a new server joins the load balancer only after your pipeline’s ready report. So the readiness deadline has to cover all of these: the boot, the event, the time the run waits in its queue, the deploy, the report and the health check.

A server that stopped in “Deploying” got no ready report. Check two things:

the deliveries on the group’s Deployment page, each with its state and last answer. A failed delivery is retried for about 38 minutes (what counts as delivered);
your pipeline’s run.

If your queue and deploy need longer, raise Readiness deadline: before the Rehearsal on the Go Live page, up to 45 minutes, and once the node group is Live under Advanced in Settings, up to 60 minutes. How the reports work is on Webhook adapter.

Getting help

Report a problem is in the More menu (the three dots) on the node group’s Overview. It sends support:

your words, and your name and email address;
the organization and the node group, with the group’s state and generation;
the group’s last twenty timeline events, with their phase changes and reasons;
its last ten attempts to create a server, with the server type, location, outcome and Hetzner’s error code.

Each event and attempt carries its time in UTC. The reply comes to your address. A member can send up to five reports an hour. After that, write to support@hetscale.com.

By mail, name the organization, the node group, the time in UTC and the event or the server’s name, such as hetscale-web-7.

Support never asks for a Hetzner token, a Hetscale API token, a password or an SSH key. Do not send one, by mail or in a report. What support covers and when it answers is in the terms.