Storage & Backups

Your database lives in server/data/ as a tree of data.json files. The server keeps the entire tree in memory and writes dirty subtrees back to disk on a 5-second debounce. The timer flush serializes only the top-level keys written since the last flush (see FLUSH_CACHE_MB) and hands the file writes to a background thread, so it no longer stalls clients; durable writes, backups and shutdown still flush synchronously so their on-disk guarantee is unchanged. You don't normally need to touch these files — but it helps to know how they're laid out.

The split tree

To keep individual JSON files from growing unbounded, NukeBase splits any object that would exceed 128 MB serialized (SPLIT_THRESHOLD_MB) into a subdirectory. The parent file references the child via a $split marker:

server/data/
├── data.json                  # { "users": "$split", "posts": [...] }
└── users/
    ├── data.json              # { "alice": {...}, "bob": "$split" }
    └── bob/
        └── data.json          # { ...bob's subtree... }

Splits happen automatically when an object grows past the threshold, and the inverse — coalescing back into the parent file — happens automatically on startup if a child has shrunk under the threshold. Migration from a legacy single server/data.json file is also automatic on first run.

Only keys that are safe folder names get their own folder. A split child is stored in a directory named after its key, so a key is split out only if it is 200 bytes or shorter, contains none of / : * ? " < > | or control characters, is not . or .., does not end in a dot or space, is not a Windows device name or data.json, and does not differ only by case from a sibling already split. Any other key — a URL, a date like 2026/09/29, a very long key — simply stays inside its parent's file. Nothing about reading or writing it changes. (Since engine 1.5.1. Data an older engine split under such a name is folded back into its parent at startup; a split key that would point outside its own folder, such as .., stops startup like a missing shard, see Bad-disk recovery below.)

Atomic writes & crash safety

Each data.json is written via a tmp + fsync + rename sequence, so a crash mid-write leaves either the previous file or the new file on disk — never a partially-written one. Any leftover .tmp files from a crashed write are swept on startup before the data is loaded. Pending in-memory writes are flushed on SIGTERM/SIGINT for graceful shutdown.

Daily backup

Once per calendar day (UTC) the server flushes pending writes and copies server/data/ to server/backup/<YYYY-MM-DD>/. It checks hourly whether today's copy exists, starting a minute after boot, so a server that restarts often still gets one. The copy is written under a temporary name and renamed into place, so a crash mid-copy never leaves a half-written directory posing as a finished backup.

Each backup is a full copy of the database, so they are pruned: after every successful backup, dated directories older than BACKUP_KEEP_DAYS (default 7) are removed. Only directories named like a date are ever touched — anything else you keep in server/backup/ is left alone. Set BACKUP_KEEP_DAYS=0 to keep every backup.

Where files live

The file store is separate from the JSON tree and does not participate in any of the above. Files live under server/files/ (override with FILES_DIR), laid out as a plain directory tree that mirrors the path arrays you address them with — ["posts","p1","hero.png"] is server/files/posts/p1/hero.png.

Uploads use the same tmp + fsync + rename discipline: bytes land in a $tmp- prefixed file beside the destination and are renamed into place, so a reader sees the old file or the new one and never a partial write. Because a leading $ is a rejected path segment, an in-flight upload has no addressable path. Leftovers from a crash are swept at startup.