Add server hibernation

[jmunckhof]

Closes #50.

Adds a hibernate / wake lifecycle so an idle game server can stop paying compute. Snapshots the disk to Hetzner image storage, deletes the VM, and restores it on demand.

Summary

• New desiredState: hibernated, observed states hibernating / hibernated / waking
• Two new workflows: hibernateServer (drain agent → optional IP reserve → shutdown → snapshot → delete VM) and wakeServer (create VM from snapshot → wait for agent → drop snapshot)
• New server actions: hibernate, wake, plus a hibernate-settings action to toggle "reserve IP across hibernate"
• Settings panel: toggle for reserving the public IPv4 (off by default; ~€0.50/mo while hibernated)
• Activity stream surfaces the new phases through emitActivity

Cost shape (rough)

• Hibernated: ~€0.011/GB/mo snapshot storage, plus ~€0.50/mo if reserve-IP is on
• Compared to a running CCX13 at ~€8.21/mo, that's roughly a 95%+ saving while idle

Notes for reviewer

• prisma/schema.prisma changed but I haven't generated the migration yet — happy to add one before this comes out of draft
• Hibernate / wake actions atomically claim the state transition with a conditional updateMany so a double-click can't fan out two workflows; both roll back the DB if the workflow start itself throws
• Snapshot polling has an explicit timeout; on timeout the workflow throws FatalError instead of falling through to VM deletion (so a stuck snapshot can never silently nuke the world)

Out of scope

• Auto-suspend after N minutes idle (sits on top of this primitive)
• Cost preview in the UI
• Cross-region restore (Hetzner snapshots are region-locked)

Test plan

• Generate + apply Prisma migration
• Hibernate a running server end-to-end, verify VM is gone and snapshot is retained
• Wake from hibernation, verify the agent reconnects and the snapshot is cleaned up
• Toggle reserve-IP on, hibernate, wake, confirm same IPv4 returns
• Toggle reserve-IP off, hibernate, wake, confirm IP is allowed to change
• Try hibernating an unprovisioned server / a server in hibernating state, confirm the action rejects cleanly
• Force a snapshot timeout (mock or short deadline) and confirm the VM is NOT deleted

[vercel]

@jmunckhof is attempting to deploy a commit to the Hayden Bleasel Team on Vercel.

A member of the Team first needs to authorize it.

[haydenbleasel]

Nice work on this — the overall shape (atomic state claims, FatalError ordering so a stuck snapshot can't nuke the VM, idempotent step keys) is solid. A few things worth tackling before this comes out of draft:

Correctness

• Snapshot is taken before the VM is confirmed off. stepShutdownHetzner returns immediately after firing the ACPI shutdown action, then stepCreateHibernationSnapshot runs against a still-draining VM. Hetzner recommends snapshotting an off VM — risk of an inconsistent image (i.e. silently corrupt game saves, which defeats the point). Suggest a poll loop on stepGetHetznerStatus until off before snapshotting, mirroring the existing snapshot-status loop.
• Reserved IP is never released when the toggle is turned off. setHibernateSettings only flips the flag. If a user reserves, hibernates, wakes, then turns the toggle off, primaryIpv4Id stays set and Hetzner keeps charging for the reservation. stepReleaseReservedIpv4 only runs on teardown today. Either release in the settings action (when flipping true→false) or at the top of hibernateServer when the flag is off but a reservation exists.
• Wake workflow plows on after a Hetzner-running timeout. If the VM never reaches running within MAX_HETZNER_WAIT_SECONDS, the loop falls through with runningIp = null and we still call stepMarkHetznerRunning and wait for an agent that will never connect. The symmetric path in hibernateServer throws FatalError on snapshot timeout — do the same here.
• stepMarkAwake ignores intent. Unconditional updateMany({ where: { id: serverId } }) will clobber a Stop/Delete that landed during the reconnect wait and force observedState=running, phase=ready. Either gate on desiredState === \"running\" in the where clause or add an isCancelled check immediately before it.

Smaller stuff

• stepCreateHibernationSnapshot short-circuits on any existing hibernationImageId, even if the prior snapshot ended up unavailable. Consider verifying status before reusing.
• The // 422 = server already off comment in stepShutdownHetzner is probably wrong — Hetzner typically returns 409 Conflict for that case. Worth widening the accepted-status set or confirming against a real call.
• setHibernateSettings does a layout-scoped revalidatePath on every toggle flick; consider scoping it to the server path.
• Schema column alignment in Server is off vs. surrounding fields (cosmetic).

Things that read well

• updateMany + state-in-where for the action claim, with DB rollback if start throws — exactly the right pattern for double-click safety.
• FatalError on snapshot timeout before the delete-VM step.
• Reserve-IP cost trade-off is surfaced in both the toggle copy and the confirm dialog (and the dialog adapts based on the flag).

Test plan looks comprehensive; only thing I'd add is a cheap unit test on the action-level guards (e.g. "rejects when observedState=hibernating") since those are easy to regress.

[jmunckhof]

Updated this on top of the provider refactor that landed in main. Pulled out the IP-reservation toggle for now
since it was Hetzner-specific raw API calls that don't fit the new Provider abstraction cleanly. I'll do that as
a follow-up PR

Rest of the flow is the same: STOP → shutdown → snapshot → delete VM on hibernate, then create from snapshot →
wait for agent → delete snapshot on wake. Snapshot is also cleaned up if you delete a hibernated server

[haydenbleasel]

Heads up @jmunckhof — all merged now but had to make some small changes!

What changed:

Rebased onto current main. The branch predated the Ghost 2.0 single-tenant rework (#76), so your four commits were squashed into one (authorship preserved) and ported: actions moved to app/(app)/[id]/actions/, userId scoping and getProviderForUser replaced with the single-tenant getProvider(), and the schema/interface conflicts resolved against the new layout. The unrelated form.tsx htmlFor tweaks were dropped.

Added a hardening commit on top, addressing review findings:

• A snapshot timeout no longer deletes the VM and snapshot (this was the "silently nuke the world" path the PR description intended to prevent) — it marks the server failed and leaves everything for a retry to reuse.
• The workflow now waits for the VM to reach off (ACPI shutdown, then hard poweroff) before snapshotting, instead of imaging a running disk.
• Hibernate/wake actions atomically claim the state transition with a guarded updateMany and roll back if the workflow fails to start — matching what the PR description promised.
• Failed hibernations/wakes are retryable instead of dead-end states; a wake retry reuses a surviving VM, and a VM that 404s mid-wake has its dangling providerServerId cleared.
• UI: pending resets in finally, and hibernating/waking were removed from PROVISIONING_PHASES since the provisioning checklist can't represent them.
• Added the Prisma migration for the new enum values and columns.

The reserve-IP setting mentioned in the PR description was never part of the diff and remains unimplemented — the dialog copy correctly says the IP changes on wake. Thanks for the feature!