Backups and Recovery

Sync is not a backup. Sync propagates both desired changes and mistakes; a backup is an independent, restorable copy with a known retention policy.

flowchart TD
  A[Live encrypted account] --> B[User export]
  A --> C[Desktop automatic backup]
  A --> D[Email or Nextcloud encrypted backup]
  E[Database and file volumes] --> F[Infrastructure backup]
  B --> G[Restore drill]
  C --> G
  D --> G
  F --> H[Whole-service restore drill]

Backup layers

Use at least one user-level export and one infrastructure-level backup for a self-hosted deployment.

Manual account exports

The client supports encrypted and decrypted data exports and import. Prefer an encrypted export for routine retention:

1. Run a full sync.
2. Export the account data.
3. Move the file to independent storage.
4. Record the client/server version and export date.
5. Verify the archive can be opened or imported in an isolated test account.

A decrypted export should be treated as sensitive plaintext. Encrypt the storage medium or place the export in a separately encrypted archive, and delete temporary copies after verification.

The srn-client export command can also create JSON or Markdown exports. Its output is decrypted and therefore inherits the same plaintext handling rules.

If account recovery is enabled, keep its code outside the account and outside the only backup archive. Backing up the server-side escrow without the code is not sufficient; putting the code beside the escrow removes the intended separation. A code suspected of exposure must be replaced from Preferences -> Security -> Account recovery.

Desktop automatic backups

The desktop client exposes separate backup paths:

• automatic encrypted text backups;
• optional plaintext backups;
• attachment/file backups; and
• a cross-platform backup decryption drop zone.

Keep the destination outside application data so uninstalling the client does not remove the only copy. For stronger resilience, replicate the folder to an offline or separately authenticated destination.

Email backups

Email backups are available only when the server enables them and the account is entitled/configured. They are scheduled encrypted artifacts, not plaintext messages containing every note.

Before depending on email delivery:

• verify EMAIL_BACKUPS_ENABLED is effective;
• confirm outbound mail works;
• inspect the received attachment;
• understand mailbox retention and attachment-size limits; and
• complete a test decrypt/import.

Nextcloud/WebDAV backups

Nextcloud backups use encrypted server-generated backup blobs uploaded over WebDAV. Both the server master switch and complete per-user settings are required:

• Nextcloud base URL;
• optional destination folder (blank means the WebDAV account root);
• dedicated app password; and
• backup frequency.

Create a low-privilege Nextcloud app password rather than reusing the main account password. The app password is sensitive server-side configuration and is not returned by the administrator settings API.

The client validates the complete configuration before writing any setting. If an existing schedule is being edited, it disables that schedule first, saves the HTTPS URL, folder, and optional replacement app password, and writes the requested frequency last. A failed intermediate write therefore leaves the schedule disabled instead of activating a mixed old/new configuration.

flowchart LR
  A[Validate complete configuration] --> B[Disable an active schedule]
  B --> C[Save HTTPS URL and folder]
  C --> D[Save replacement app password]
  D --> E[Enable the requested frequency last]

Each upload attempt has these transport boundaries:

• the base URL is parsed and canonicalized once and must be the final HTTPS origin, without embedded credentials, query, or fragment;
• the account identifier, every supplied folder component, and the generated file name must be safe path segments. A completely empty folder preserves the established account-root destination, while leading, trailing, or interior empty components and dot, dot-dot, backslash, or control-character components are rejected;
• every DNS answer must be public, then one validated address is pinned to the outbound socket while the original host remains the HTTP Host and TLS SNI identity. Private, loopback, link-local, and metadata destinations fail closed;
• redirects are never followed or counted as WebDAV success, so the Basic credential cannot be replayed to another origin;
• one 60-second absolute deadline covers DNS resolution, every nested MKCOL, the PUT, and response-body draining; and
• MKCOL accepts only 201 (created) or 405 (already exists), while PUT accepts only 200, 201, or 204.

There is no automatic retry inside an upload attempt. The scheduled cadence advances only after the syncing service reports a confirmed upload completion; merely publishing the asynchronous request does not count as success. Failed requests remain due. A request stays in-flight for at most 30 minutes, then failed or expired work retries through a bounded exponential backoff from 15 minutes up to 6 hours, so a frequent scheduler cannot create an unbounded event storm.

Domain events are delivered at least once. A redelivered request reuses the original event’s validated UTC date and uploads the same deterministic SN-Data-YYYY-MM-DD.json path—even when redelivery crosses midnight—so it overwrites that encrypted artifact instead of creating an unbounded set of duplicates. Completion truth is monotonic: a confirmed success always wins; an earlier failed receipt may be upgraded by a later success for the same request, while a later failure cannot downgrade a recorded success. This matters when queue visibility expires while a 60-second WebDAV attempt is still running.

Auth serializes each user’s claim, completion history, and last-success cadence through a database transaction shared across workers and processes. The state and NEXTCLOUD_BACKUP_LAST_RUN mirror commit together, so overlapping scheduler passes cannot double-dispatch and out-of-order completions cannot regress the cadence or overwrite another active request. Lock rows are deleted with their user; a late completion for a deleted account is terminally discarded rather than retried forever. Completion history is bounded. Lifecycle state and completion receipts contain request identifiers, outcome, and timestamps only—never the Nextcloud URL, app password, or raw provider error text.

Outside direct-call home-server mode, credential-bearing NEXTCLOUD_BACKUP_REQUESTED events use a dedicated SNS topic configured by NEXTCLOUD_BACKUP_SNS_TOPIC_ARN. It must be a valid SNS topic ARN, must differ from the general auth topic, and must have exactly one SQS subscription: the syncing worker queue. Auth, files, websocket, analytics, and other consumers must not subscribe. Auth validates that the ARN is present, structurally valid, and different from its general topic; it cannot inspect an external topic’s subscription inventory. A missing, malformed, or general-topic ARN fails backup dispatch closed, and auth never falls back to the general topic. The bundled Compose/floci topology provisions the isolated route automatically.

If the syncing queue uses a dead-letter queue, the redrive path can legitimately carry the same credential payload. Protect that DLQ to the same standard: server-side encryption, narrowly scoped access, short and audited retention, no broad inspection/export tooling, and a documented purge procedure. Rotate affected app passwords after any uncertain queue or DLQ exposure.

Rolling upgrade compatibility

This release adds both the dedicated credential topic and transaction-backed lifecycle locks. Do not perform an ordinary unconstrained rolling deployment.

Use this order for bundled or external multi-service deployments:

1. Set the persisted nextcloudBackupsEnabled override and NEXTCLOUD_BACKUPS_ENABLED to false, then stop every daily, weekly, and monthly Nextcloud cron. Confirm no scheduler can start a new request.
2. Apply the auth database migrations. Verify nextcloud_backup_user_locks.user_uuid is a primary key with a foreign key to users.uuid using ON DELETE CASCADE.
3. Create the dedicated SNS topic. Subscribe only the syncing SQS queue, grant auth publish permission and syncing consume permission, and set NEXTCLOUD_BACKUP_SNS_TOPIC_ARN on every new auth server and worker. Keep the legacy general-auth-topic subscription on syncing temporarily so queued requests from old auth can drain.
4. Deploy syncing first and verify it consumes both the legacy route and the dedicated route. Use a credential-free topology probe or subscription inventory; do not place a real app password in a test message.
5. Drain or stop every old auth server, worker, and scheduler. Confirm none can publish or write lifecycle state, then deploy the new auth version. A new auth worker refuses backup dispatch when the dedicated ARN is missing, malformed, or equal to the configured general auth topic. Separately verify the external subscription inventory; auth cannot detect reuse of a different multi-subscriber topic.
6. Inspect the auth, files, websocket, analytics, and other unrelated queues for pre-fix backup events according to the operator’s secure queue procedure. These are shared queues: never purge an entire queue casually. Drain or filter backup events with an audited operator procedure that preserves unrelated messages, or explicitly accept and document the unrelated-message loss before a whole-queue purge. Inspect the syncing DLQ without exporting payloads, remove retained credential copies safely, and rotate affected app passwords when exposure cannot be ruled out.
7. Run one controlled backup, verify the deterministic Nextcloud artifact and confirmed completion, then re-enable the persisted/environment master gate and cron schedules.

The mixed-version behavior while the gate is paused remains compatible:

• An older auth service emits backup requests without acknowledgement IDs. The newer syncing service still uploads them and derives a stable UUIDv5 from the original event timestamp, user UUID, and correlation identity. Queue redelivery therefore uses the same acknowledgement identity and the same daily destination. The older auth service ignores the completion receipt and retains its established publication-based cadence.
• A newer auth service sending to an older syncing service still gets an upload, because the older consumer ignores the additional request ID. The old consumer cannot send a completion receipt, so new auth does not claim a confirmed success: the request expires after 30 minutes, enters bounded backoff, and remains retryable. Once syncing is upgraded, a retry completes the acknowledgement. Repeated attempts overwrite that day’s deterministic artifact rather than creating extra files.

Do not downgrade services independently after the new private delivery state or lock rows have been written. Pause the gate and cron again, preserve the database, roll back auth and syncing together, restore the legacy event topology only if the older pair requires it, and verify an actual Nextcloud artifact before treating the backup as successful.

flowchart LR
  A[Persist bounded in-flight request] --> B[Upload encrypted daily artifact]
  B -->|Confirmed completion| C[Advance last-success cadence]
  B -->|Failure| D[Keep backup due]
  A -->|No completion in 30 minutes| D
  D --> E[Retry after 15 min to 6 h backoff]
  E --> A

Verify and recover a Nextcloud backup

After configuration, confirm a new SN-Data-YYYY-MM-DD.json file appears, then perform a restore drill with a disposable account:

1. Download the newest artifact without modifying the Nextcloud copy.
2. Record its size and checksum, and preserve the original during the drill.
3. Import it through the application’s backup import flow using the account password required to decrypt its item payloads.
4. Verify representative notes, tags, and expected metadata. This current-item backup does not replace revision-history or attachment/file-volume backups.
5. Revoke the drill credential and remove disposable restored data according to the retention policy.

If an upload fails, keep scheduling disabled while checking the final URL, public DNS answers, TLS certificate, destination folder, server master switch, per-user administrator opt-in, and app-password permissions. Replace rather than reuse a credential after any suspected disclosure. Server logs expose a stable failure category but do not include the URL or app password.

Self-hosted infrastructure backups

Back up these together:

• MariaDB/MySQL data;
• Redis only if the deployment deliberately relies on persistent Redis state;
• encrypted file-storage volumes or object storage;
• the server-data gateway settings and encrypted subscription-pairing store;
• the exact secrets needed to validate sessions and decrypt protected configuration; and
• deployment manifests and image/version identifiers.

If upgrading a Compose installation that still has an initialized mysql-data volume, create a logical dump with the old MySQL service before starting MariaDB. A raw MySQL datadir is useful only as a protected recovery copy for the matching MySQL engine; it is not a MariaDB restore artifact. Follow the legacy MySQL volume migration and keep the dump plus the untouched legacy volume until the MariaDB restore has passed application-level checks.

Do not store the only copy of .env beside the live host. Protect database credentials, JWT secrets, valet secrets, shared access keys, SMTP credentials, provider keys, subscription-pairing encryption keys, and backup app passwords. Keep the pairing key separately from the only server-data backup: the encrypted pairing file cannot be recovered without both.

The repository’s yarn ops:backup-restore gate exercises the scripted backup and restore contract. It is validation evidence, not a substitute for restoring your own production-size data on your own storage.

Restore order

For a whole-server recovery:

1. Stop application writes.
2. Preserve the failed state for investigation.
3. Restore configuration and secrets.
4. Restore the database.
5. Restore encrypted file storage.
6. Start dependencies, then application services.
7. Run liveness and readiness checks.
8. Sign in with a test account and open representative notes, revisions, and files.
9. Re-enable external traffic.
10. Confirm scheduled backups run again.

Follow the exact commands in Self-Hosting and Operations Hardening for the deployed profile.

Recovery drills

A useful quarterly drill proves:

• the backup exists outside the live failure domain;
• checksums or archive integrity pass;
• the required password and secrets are available;
• notes, tags, revisions, and representative attachments open;
• a restored client can sync without overwriting production; and
• the measured recovery time meets the service objective.

Record the date, backup identifier, restore duration, gaps found, and corrective action. A backup that has never been restored is only an assumption.

Include account recovery in a drill only with a disposable account: enable it, save the code outside the account, sign out, recover with a new password, save the replacement code, verify the old code fails, and then disable recovery if the production policy does not require it. Never use a production account as the first recovery test.

Retention and deletion

Live-account deletion does not remove independent copies. Define retention for desktop folders, exports, email, Nextcloud, database snapshots, object storage, and disaster-recovery replicas. When a deletion request must cover backups, document whether the copy is immediately purged or expires through normal rotation.

For migrations to or from original Standard Notes clients and servers, review the evidence and test matrix in Standard Notes Compatibility.