feat: workspace split, the self-hosted sync server, and one-command install - #36
Merged
Conversation
…l writes Strips the SSH sync provider out of Daily entirely and reshapes the remote-storage contract so a provider can declare that it supports revision-aware conditional writes. SyncEngine gains a read-merge-write-retry loop that runs only against a provider declaring the capability: it reads the revision, merges, writes conditionally, and on conflict re-reads, re-merges and rewrites, bounded by SYNC_CONFIG.conditionalWriteMaxAttempts. Every attempt merges through _pull, so a successful retry leaves the other device's work in the local database and not only in the snapshot it pushed. Exhausting the attempts is a deferral rather than a failed cycle — it is recorded in that remote's state and the status stays active, because a lost race is ordinary in multi-device sync and a red status has to keep meaning that sync is broken. Against a provider that does not declare the capability — iCloud and the CLI's sync folder — behaviour is unchanged. Revision support extends IRemoteStorage rather than adding optional members to it, so an adapter cannot half-implement it. Snapshot.version is not bumped: a revision is transport metadata held by the provider, never a field inside the snapshot. Removing SSH takes the adapter, the three renderer settings fragments, SshSyncSettings and the ssh field of SyncSettings, leaving sync iCloud-only with no compatibility shim. Migration v009 strips the dead ssh key from stored device_settings, guarded by json_valid so an unreadable row is skipped rather than aborting the migration. No provider declaring the capability exists yet; the contract and the engine loop are verified against a fake.
Adds src/server/ as a fourth source root — its own tsconfig, vite build and vitest project, and the daily-server console — and gives it the half of the Daily Sync Protocol that decides who is talking. A server identifies itself over an unauthenticated handshake and reports whether anyone has claimed it. A six-digit code its console prints binds the first device exactly once, behind an attempt cap that locks the code and a console command that regenerates it. Every device after that is bound by peer approval: the new device asks, any already-bound device reads the code and the name it was shown and confirms or declines, and the approved device collects its credential inside a bounded pickup window. When no device survives, the console issues a single-use token that binds a fresh one. Device credentials are stored only as sha256 hashes and authenticate every request. Revoking one sets revoked_at, leaves the row in place, and refuses that device from its next request onwards — distinctly from a token that was never issued. The protocol's request and response shapes, its error codes and their HTTP statuses live in src/shared/, so the desktop client can import the same contract later. No task data crosses this protocol yet: there is no snapshot endpoint, no asset channel and no revision probe, and nothing in the Daily app changes.
…rotocol Give the Daily Sync Server the half of the protocol that carries the data: one snapshot it stores and hands back with the revision it was read at, a conditional write that accepts an overwrite only from a client that had read the current state, a cheap probe that says whether that revision moved, and an asset channel for attachment bytes. Plus one console command that shows the owner what the server is holding. A bound device can now read the snapshot, merge locally, write it back conditionally, lose the race and be told so, poll a two-field endpoint, and push and pull attachment bytes — all over the authenticated request the enrollment work delivered. The server originates nothing, merges nothing, and resolves no conflict: it stores what a client pushes and hands back what it stored. The conditional write is one compare-and-swap inside a single immediate transaction, so two connections racing at the same expected revision produce exactly one winner. Assets stream to a temp file under a byte cap checked ahead of each chunk, hashed as they go and renamed into place only on a clean end, so a half-received upload is never visible. Snapshot bodies may be gzipped in both directions, bounded compressed and decompressed. Route matching now runs before the request body is read, which is what makes the per-route body cap possible. One consequence, taken deliberately: a POST to a GET-only path answers 405 rather than reporting a problem with a body that route would never have accepted. The desktop app is untouched — no renderer, main-process, IPC, storage or CLI change. src/server stays free of Electron and of src/main.
Daily gains a second bidirectional sync provider beside iCloud: the person's own Daily Sync Server. A remote adapter declares revision support and speaks the wire contract entries 2 and 3 built, so the engine drives it through the conditional-write loop that already exists and nothing about merging changes. A connection wizard binds this Mac with the claim code a fresh server printed, or by showing a code that a peer approves; that peer sees the request in a dialog of its own, carrying the same code and the asking Mac's name, with Approve and Decline. A self-signed certificate's fingerprint is shown once and pinned, and a later certificate that does not match is refused inside the connector before any request body is written, so the device credential never reaches an impostor. Plain HTTP is free on a private address and takes a deliberate confirmation on a public one, and either way the binding is labelled insecure for as long as it lasts. A twelve-second revision probe turns a change made on another Mac into a sync cycle on this one, and a revoked credential stops both the probe and auto-sync while leaving the binding in place, so the section can say what happened. Two invariants hold by construction rather than by convention. At most one bidirectional provider is ever built: binding is refused while iCloud sync is on, and enabling iCloud is refused while a server binding is active. And no device credential crosses the IPC boundary — every renderer-facing type is a view that omits it, including the settings channel that predates this work. The three-position provider control and the migration flow are entry 5 and are not built here; the server provider is reachable from a section of its own beside the iCloud switch, and refuses to bind while iCloud is on rather than inventing that migration.
Replace the two independent sync switches with a single three-position control — Off, iCloud, Self-hosted Daily — so the two can no longer both be writable at once. A single guard on StorageController.saveSettings enforces that on every path a settings write actually takes, and holding a server credential is now separate from the server being active, so a credential can be obtained while iCloud is still the provider. Changing position is a migration, not a settings write. It verifies the target, reports what each side holds and how they differ, takes a direction for records changed in both places, merges through the one existing SyncEngine, and moves the provider only once that merge has landed on both sides. A migration that fails rolls back and leaves the person on the provider they started on, still syncing. The direction decides ties and nothing else: both sides converge to the union, and it is asked about only when the preview finds a record the two sides genuinely contest. Merging such a tie in the target's favour never reached the local database — hasChanges gated the write on updated_at, which a tie has in common by definition — so keep-target silently did nothing. The merge now carries those documents through, and a converged pair still writes nothing. Also: revocation is a typed fact learned where the probe already learns it, rather than a substring matched in an error message, and it reaches the open settings pane as a broadcast; one narrowing produces the renderer's view of a binding, so the device token cannot leak through a second one that disagrees; and the six-digit code length is one constant.
Ship the Daily Sync Server as one artifact: a multi-arch image on GHCR, brought up with a copied compose.yaml and `docker compose up -d`. Configuration comes only from the environment — there is no configuration file, no first-run wizard, no install script, no systemd unit and no self-update. What the wizard used to be worth is kept as behaviour rather than as a conversation: while the server is unclaimed it verifies its own public address on every start and prints the result beside the claim code, naming the reason when it fails and never blocking the listen. `daily-server verify` performs the same check on demand. A server told to use a self-signed certificate mints one for the public URL's host, reuses it across restarts, and refuses to start rather than replace a certificate covering a different host. Pushing a server-v* tag publishes the image under three tags: the exact version, latest, and a rolling p<N> whose N is read from SYNC_PROTOCOL_VERSION in the source. The client refuses a protocol mismatch on exact inequality, so the tag is derived and never typed. The Daily app on the Mac stays the administrative surface. Recovery and inspection stay where they already are — status, claim-code, device enroll|list|revoke — reached through `docker compose exec`. The compose stand under tests/server/stand/ drives both topologies: a proxy stand running deploy/compose.yaml verbatim behind caddy, and a vps stand with the commented ports and self-signed pair uncommented. Key material is regenerated by mkcerts.sh and stays out of the tree.
The install path sent people into a macOS app repository to fetch one file. Nothing about the delivery needs the repository, so it no longer appears in it: the compose file is inlined in the README to save as compose.yaml, and a single `docker run` covers trying the server out and running it on a bare VPS. The Dockerfile carries OCI labels, so the GHCR package page has a title, a description and a link back to the source instead of being an orphan. The version label comes from DAILY_SERVER_PACKAGE_VERSION, like the package version already does. Inlining the compose file makes a second copy of it, so a test pins the README's yaml block to deploy/compose.yaml byte for byte — the copy people paste cannot drift from the one the stand drives.
Adds the Daily Sync Protocol v1 server under src/server, the desktop provider that talks to it, and the one-provider settings migration. Removes the SSH provider in favour of revision-aware conditional writes. The server ships as a multi-arch image on GHCR, configured entirely from the environment, and is brought up with a single docker run.
The boundaries between the protocol, the generic utilities, the storage core and the four programs are package dependencies now, not path aliases anyone could follow. The desktop app, the CLI and the server each declare what they use, so none of them can reach into another's internals and each installs on its own. The storage core no longer knows about SQLite or Electron. It takes a driver through a port, and every runtime brings its own: better-sqlite3 in the app, node:sqlite in the CLI, and the server's own copy widened for BEGIN IMMEDIATE. Nothing the product does changes.
Drops apps/cli and every seam it left behind: the mutation-signal file and its watcher, StorageController.handleExternalDataChange, FolderRemoteAdapter, the launcher the packaged app wrote into Resources and the settings row that installed it, and the npm publish step in the release workflow. The two-node convergence tests move onto ICloudRemoteAdapter, which takes the same directory argument and keeps all three last-write-wins cases intact.
The workspace split left every shipping path aimed at the pre-split tree: CI type-checked directories that no longer existed, the image copied src/, and the packager read a stale root out/. The app, the server and the website now build, version and release from their own packages.
Behind a reverse proxy the server starts before the proxy joins its network, so the first check always failed. It now retries while the server is unclaimed, so the address is confirmed without a restart.
The topology was prose, so both files were left to the reader. The order matters because Compose refuses a network it did not label. The nginx route sets client_max_body_size: its 1 MB default breaks attachments while the rest looks healthy.
The address step asked for a server address and said nothing about where to get a server.
Self-hosting meant pasting a compose file, following a proxy recipe and choosing between three TLS modes. One script now asks for a domain and picks the topology from the machine. Dropping own-certificate leaves the server two transports instead of three.
Inline run: blocks cannot be linted or tested, and get forgotten. Nine move to scripts/ci/, and a test now fails any multi-line run: step. A second test ties install.sh's pinned p<N> to SYNC_PROTOCOL_VERSION, so a protocol bump can no longer leave new installs on the old image.
pnpm-workspace.yaml uses allowBuilds, added in pnpm 10.26, but CI pinned 10.12.1. That version ignores the key, so no package got build approval, electron's postinstall never ran and the test job died on a missing binary. Local pnpm is 10.30.1, which is why this only failed on CI.
deepMerge assigned every source key straight onto the target, so a payload with a __proto__ key polluted Object.prototype. It merges settings read back from the sync server, so the payload is not always ours. Skip __proto__, constructor and prototype instead.
Imports in the AI tool registry reached back four levels. vite-tsconfig-paths was already wired into all three builds, so only the tsconfig paths were missing. Adds @main/* and @/*, and moves the 22 formatters imports over. Relative imports still work, so the rest can follow later.
Rewrites the remaining 422 deep relative imports. vite-tsconfig-paths does not resolve aliases imported from .vue files, so main, preload, renderer and vitest now declare the aliases explicitly. Adds @shared for the cross-process directory. ESLint ignored only the root out/, not apps/desktop/out/.
Packages ship raw TypeScript, so an alias used in src has to resolve in every consumer's build too, and typecheck:main broke on it. Tests are not consumed by anyone, so the alias stays inside the package. Source keeps relative imports.
The earlier pass only rewrote imports three levels up or more, so one file could mix @/stores/theme.store with ../../base/BaseButton and nothing explained the difference. The rule is now whole: ./ stays relative, anything above it goes through an alias.
The workspace split moved dev into apps/desktop, so pnpm dev from the root stopped working. The root already proxies build:server, test and lint the same way, so dev was the odd one out.
App.vue asks for a pending approval on every start, so a device with sync off logged a NO_BINDING error each time. Asking whether anyone waits is a question, and "no server" is a fair answer to it. approve and deny still throw, because calling those unbound is a mistake.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings
mainup to the workspace layout and the self-hosted sync server. 15 commits, 1014 files.Read this before reviewing: it is two distinct bodies of work, and only the last two were verified by the run that produced them.
1. The workspace split and the sync server — 13 commits, predating this run
e7138700throughf8a8e424:src/split into seven packages behind a driver, the CLI removed, each artifact shipped from the package that owns it, the Daily Sync Server stood up with protocol v1, sync through a self-hosted server, the server delivered as a Docker image, the SSH provider removed.These were the base this run built on. They did not pass through its gates — no code review or runtime verification here covers them. They need reviewing on their own terms.
2. One-command install — 2 commits, gated
feat(server): install the sync server with one command—deploy/install.shand its three topology generators, the generateddaily.shwith eight management verbs, adaily-server healthchecksubcommand, the transport cut from three modes to two (own-certificateremoved), and the server README rewritten from 302 lines to 123.ci: move workflow scripts out of YAML and pin the protocol tag— nine inlinerun:blocks extracted toscripts/ci/, a test that fails any multi-linerun:step, and a guard tyinginstall.sh's pinnedp<N>toSYNC_PROTOCOL_VERSION.All three install topologies, the healthcheck and the management verbs were driven on a real Docker host, not only unit-tested. Per-case evidence is in
.ai-workflow/run/2026-09-07-one-command-server-deploy/gate-b/.The image was built from this branch for those runs — the published
ghcr.io/scheron/daily-server:p1does not exist yet and would predate thehealthchecksubcommand the generated compose depends on. So those results prove the code, not the distribution.Two things block a working release, neither of them code
server-v*tag has ever been cut, sorelease-server.ymlhas never run and the image was never published.Until both hold,
install.shfails atdocker compose pullfor every operator, not just onupgrade. That is the one runtime case that did not pass; every step after the fetch was verified by rebuilding the tag locally.The README's install URL also 404s until this merges, since it points at
main.Known and not fixed
daily.sh backupwrites the database at the process umask — 0644 besidedaily.shon a root install. Oneumask 077closes it.backupleaves the server stopped: underset -eua full disk aborts before the restart, silently.shellcheckhas never run againstinstall.shlocally — it is not installed there, so the new CI job runs for the first time on this PR. SC2154 is plausible on the five scripts reading CI-supplied variables; the usual# shellcheck disableremedy is a comment, which this project forbids, so a structural form would be needed.run: cmd-a && cmd-bon one line still passes it. Nothing here does that.Full decision log for the gated part, including two plan conflicts resolved mid-run, is in
.ai-workflow/plans/2026-09-07-one-command-server-deploy.mdunder## Corrections during execution.