GitHub Actions
Last updated
Two workflows for a node group on the webhook adapter, built from the Hetscale Deploy action,
hetscalehq/deploy-action, listed on the GitHub
Marketplace as Hetscale Deploy. The first
deploys a new release to the servers that run, while the group’s deploy lock holds scaling back; the
second deploys each new server Hetscale creates. Each step of the action is one call of the API, and
the action’s README lists every input and output.
What do I need?
- A node group on the webhook adapter, Live.
- An API token, made on the panel’s API tokens page by a member who may operate the group (on
Team: an operator, an admin or the owner), with the abilities Lock groups (
groups:lock), Change releases (releases:write), Read nodes (nodes:read) and Report nodes (nodes:report); for the variant that replaces the servers instead, also Replace older servers (rollouts:write). The new-server workflow can use a second token with only Read nodes and Report nodes. Store the token as the repository secretHETSCALE_TOKEN, and the group’s id (grp_…) as the repository variableHETSCALE_GROUP. - The SSH access your deploy job already has to the servers.
How do I deploy a release to the running servers?
One workflow runs on every push to main. It takes the deploy lock, records the release, reads
the serving servers, deploys to each of them as the job already does, reports Hetscale’s servers
deployed, and releases the lock:
name: Deploy
on:
push:
branches: [main]
permissions:
contents: read
# One deploy per group at a time: the lock has no owner, so this is what keeps runs apart.
concurrency:
group: hetscale-deploy-${{ vars.HETSCALE_GROUP }}
cancel-in-progress: false
jobs:
deploy:
runs-on: ubuntu-latest
timeout-minutes: 25 # below the lock's ttl of 1800 seconds
env:
IMAGE: ghcr.io/acme/app:${{ github.sha }}
steps:
- uses: actions/checkout@v7
# Build and push $IMAGE here, as the job already does.
- name: Take the deploy lock
id: lock
uses: hetscalehq/deploy-action@v1
with:
command: lock
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}
ttl: 1800
- name: Record the release
id: release
uses: hetscalehq/deploy-action@v1
with:
command: release
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}
ref: ${{ env.IMAGE }}
- name: Read the serving servers
id: nodes
uses: hetscalehq/deploy-action@v1
with:
command: nodes
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}
release-id: ${{ steps.release.outputs.release-id }}
address: ipv4 # private_ip when the runner reaches the private network
- name: Deploy to each server
shell: bash
env:
HOSTS: ${{ steps.nodes.outputs.hosts }} # outputs reach scripts through env, never ${{ }} in run
run: |
while IFS= read -r host; do
[ -n "$host" ] || continue
ssh -n "deploy@$host" "./deploy.sh $IMAGE" # the deploy your job already runs
done <<< "$HOSTS"
- name: Report the servers deployed # never with if: always(), which would report a failed deploy
uses: hetscalehq/deploy-action@v1
with:
command: deployed
token: ${{ secrets.HETSCALE_TOKEN }}
release-id: ${{ steps.release.outputs.release-id }}
node-ids: ${{ steps.nodes.outputs.managed-node-ids }}
- name: Release the deploy lock
if: always() && steps.lock.outcome != 'skipped'
uses: hetscalehq/deploy-action@v1
with:
command: unlock
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}- The release is recorded before the deploy, so every server created from then on boots it.
nodes, given the release just recorded, first waits for servers that started on an earlier release, so that they serve and are inhostswhen the deploy runs: up to itswaitinput, 600 seconds by default. Any still starting at the end are named inin-flight-older, with a warning, and stay older until a rollout or the next deploy.- The deploy reaches every serving server, the seed included.
deployedreports only Hetscale’s own servers, and skips one that left or failed meanwhile, with a warning. deployednever runs withif: always(), which would report a failed deploy. If the deploy fails half-way, run the job again, or make the earlier release current on the group’s Deployment page.- Outputs reach scripts through
env:, never as${{ }}insiderun:. In a public repository, the deploy step’senv:, with the addresses, shows in its log. - Servers without public addresses have
ipv4: null: setaddress: private_ipwhen the runner reaches the private network (Servers without public addresses). - Keep
ssh -n: inside awhile readloop,sshwould otherwise read the rest of the host list from standard input.
Let Hetscale replace the servers instead
This variant keeps the lock and deploys to the seed itself, because a rollout never replaces the seed, which would otherwise keep serving the earlier build. Then it releases the lock and starts a rollout: Hetscale replaces the older servers one at a time, and the new-server workflow below deploys each new one. Replace the steps from Read the serving servers to the end with:
- name: Read the serving servers
id: nodes
uses: hetscalehq/deploy-action@v1
with:
command: nodes
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}
- name: Deploy to the seed
shell: bash
env:
HOSTS: ${{ steps.nodes.outputs.adopted-hosts }}
run: |
while IFS= read -r host; do
[ -n "$host" ] || continue
ssh -n "deploy@$host" "./deploy.sh $IMAGE"
done <<< "$HOSTS"
- name: Release the deploy lock
if: always() && steps.lock.outcome != 'skipped'
uses: hetscalehq/deploy-action@v1
with:
command: unlock
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}
- name: Replace the older servers
uses: hetscalehq/deploy-action@v1
with:
command: rollout
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}rollout succeeds with an empty rollout-id when every server already runs the current release,
and with a warning when a rollout already runs.
How does a new server get the release?
Hetscale sends node.created to the group’s endpoint. GitHub starts a workflow only for a request
that carries a GitHub token, and Hetscale’s deliveries carry only its own headers, so a small relay
sits between them. As a Cloudflare Worker:
// Verifies Hetscale's node.created and starts the workflow. Secrets: HETSCALE_SECRET (the
// endpoint's signing secret) and GITHUB_DISPATCH_TOKEN; variables: GITHUB_REPO ("owner/repo")
// and WORKFLOW ("new-server.yml").
const encoder = new TextEncoder();
async function hmacHex(secret, message) {
const key = await crypto.subtle.importKey(
'raw',
encoder.encode(secret),
{ name: 'HMAC', hash: 'SHA-256' },
false,
['sign'],
);
const mac = new Uint8Array(await crypto.subtle.sign('HMAC', key, encoder.encode(message)));
return [...mac].map((byte) => byte.toString(16).padStart(2, '0')).join('');
}
export default {
async fetch(request, env) {
const body = await request.text();
if (request.headers.get('X-Hetscale-Event') !== 'node.created') {
return new Response(null, { status: 204 });
}
const timestamp = request.headers.get('X-Hetscale-Timestamp') ?? '';
if (!(Date.now() / 1000 - Number(timestamp) <= 300)) {
return new Response('stale delivery', { status: 400 });
}
const expected = encoder.encode(await hmacHex(env.HETSCALE_SECRET, `${timestamp}.${body}`));
const valid = (request.headers.get('X-Hetscale-Signature') ?? '').split(',').some((value) => {
const [scheme, hex = ''] = value.trim().split('=');
const given = encoder.encode(hex);
return (
scheme === 'v1' &&
given.byteLength === expected.byteLength &&
crypto.subtle.timingSafeEqual(given, expected)
);
});
if (!valid) return new Response('bad signature', { status: 401 });
const event = JSON.parse(body);
const github = await fetch(
`https://api.github.com/repos/${env.GITHUB_REPO}/actions/workflows/${env.WORKFLOW}/dispatches`,
{
method: 'POST',
headers: {
Authorization: `Bearer ${env.GITHUB_DISPATCH_TOKEN}`,
Accept: 'application/vnd.github+json',
'User-Agent': 'hetscale-relay',
},
body: JSON.stringify({
ref: 'main',
inputs: {
node_id: event.node.id,
ip: event.node.ipv4 ?? '',
release: event.release.id,
generation: String(event.group.generation),
},
}),
},
);
// A 2xx only once GitHub took the dispatch (204), so Hetscale retries otherwise.
return new Response(null, { status: github.ok ? 204 : 502 });
},
};- The relay answers 204 to every event other than
node.created, and refuses a delivery whoseX-Hetscale-Timestampis more than five minutes old. - It computes the HMAC-SHA256 of
<timestamp>.<body>with the endpoint’s secret and compares it, in constant time, with eachv1=value ofX-Hetscale-Signature: there are two during the 24 hours after you rotate the secret. GITHUB_DISPATCH_TOKENis a fine-grained GitHub token for that one repository with Actions read and write. It stays in the relay; Hetscale never sees it.- The relay answers 2xx only after GitHub’s 204, so Hetscale retries a delivery GitHub did not take.
The workflow it starts deploys to the new server and reports the outcome:
name: Deploy a new server
on:
workflow_dispatch:
inputs:
node_id: { type: string, required: true }
release: { type: string, required: true }
generation: { type: string, required: true }
ip: { type: string, required: false } # sent by the relay; the address comes from the inventory
permissions:
contents: read
# A repeated delivery waits for the first run, and then finds nothing to deploy.
concurrency:
group: hetscale-node-${{ inputs.node_id }}
cancel-in-progress: false
jobs:
deploy:
runs-on: ubuntu-latest
timeout-minutes: 12 # the readiness deadline (15 minutes by default) also counts the time in the queue
steps:
- uses: actions/checkout@v7
- name: Read the new server
id: node
uses: hetscalehq/deploy-action@v1
with:
command: node
token: ${{ secrets.HETSCALE_TOKEN }}
group: ${{ vars.HETSCALE_GROUP }}
node-id: ${{ inputs.node_id }}
release-id: ${{ inputs.release }}
address: ipv4
- name: Deploy the node's release
id: deploy
if: steps.node.outputs.deploy == 'true'
shell: bash
env:
HOST: ${{ steps.node.outputs.address }}
REF: ${{ steps.node.outputs.release-ref }}
run: ssh -n "deploy@$HOST" "./deploy.sh $(printf %q "$REF")"
- name: Report ready
if: steps.node.outputs.deploy == 'true'
uses: hetscalehq/deploy-action@v1
with:
command: ready
token: ${{ secrets.HETSCALE_TOKEN }}
node-id: ${{ inputs.node_id }}
release-id: ${{ inputs.release }}
generation: ${{ inputs.generation }}
# Only on a failed deploy, not on a cancel: a cancelled run leaves the node to a re-run or its deadline.
- name: Report failed
if: failure() && steps.deploy.outcome == 'failure'
uses: hetscalehq/deploy-action@v1
with:
command: failed
token: ${{ secrets.HETSCALE_TOKEN }}
node-id: ${{ inputs.node_id }}
release-id: ${{ inputs.release }}
generation: ${{ inputs.generation }}nodetakes the server’s address and its release’s ref from the inventory, never from the dispatch, and fails when the dispatch names another release than the server boots.- Its
deployoutput istrueonly while the server waits for its pipeline. Once itsreadyreport is taken, the server is a load balancer target, so a repeated delivery deploys nothing. Traffic opens only when the load balancer’s health check passes. - The readiness deadline, 15 minutes by default, also counts the time the run waits in GitHub’s
queue; hence
timeout-minutes: 12. If your queue and deploy need longer, raise Readiness deadline before the Rehearsal on the node group’s Go Live page (up to 45 minutes), or under Advanced in Settings once it is Live (up to 60), and raisetimeout-minuteswith it. readywarns, and succeeds, when the server is leaving or its deploy was cancelled; it fails for a server that failed.failedruns only on a failed deploy, not on a cancel: a cancelled run leaves the node to a re-run or its deadline.
How do I keep deploys apart?
- The deploy lock has no owner: any token with Lock groups extends or releases it.
- Run the deploy under
concurrency: hetscale-deploy-<group>withcancel-in-progress: false, as above, and lock each group from one workflow only. - GitHub’s concurrency groups apply within one repository. A lock taken or released from another repository, from Kamal or from a script shares only the TTL with the workflow.
- Release the lock in a step of its own, with
if: always() && steps.lock.outcome != 'skipped'. - Set
timeout-minutesbelow the TTL: a force-cancelled job or a lost runner runs no further step, and the lock then ends on its own. - A deploy longer than the TTL runs
lockagain before its long part, with a TTL of up to 7,200 seconds.
How do I pin the action?
uses: hetscalehq/deploy-action@v1follows the latestv1.x.y; each release is immutable.- Organisations that require SHA pinning use the full commit with its version as a comment, which
Dependabot keeps up to date:
uses: hetscalehq/deploy-action@c9ba0fd7e9fc6997377abc947c91aaf40ff8c154 # v1.0.1. - A removed or renamed input or output, or a new Node runtime, comes only in a new major version.
- The action runs on the
node24runtime: GitHub-hosted runners have it, and a self-hosted runner needs v2.328.0 or later. Behind a proxy, setNODE_USE_ENV_PROXY: 1in the step’senv:next toHTTPS_PROXY.
What about other CI?
GitLab CI or any other CI makes the same calls with curl and jq: the Kamal page’s
script shows them in order, though it reads the servers once, without the action’s wait for servers
still starting on the earlier release, and the API reference lists every endpoint. A
retry of a POST should send the same Idempotency-Key.
The webhook adapter explains the events, the reports and the deploy lock, Kamal applies the same calls to Kamal, and the API reference lists every endpoint.