Ratio-3.0

Appearance

Theme pipeline internals ​

Chapter 13. You've seen how a theme renders; this chapter is how it's stored and shipped — the content-addressed bundle, publish, and the base-rebase machinery.

How a theme is packed, stored, compiled, published, loaded, and rebased — the content-addressed bundle machinery. Code: packages/builder-core/src/theme/{bundle,assets,compile,theme-store,theme-compose,base-propagation}.ts and packages/data-objects.

Bundle format — bundle.ts ​

A theme's text files are ThemeFiles = Record<string, string> (path → UTF-8; binary bytes are not here, only their manifest entry). Packing is pure:

  • canonical(files) — a path-sorted array of [path, content] pairs, then JSON.stringify. It's an array, not an object, so a file literally named __proto__ round-trips (an object key would hit the prototype setter → silent drop → data loss + a wrong hash). Fixed order → identical file sets → identical bytes.
  • bundleId(files) = sha256(canonical).hex — hashes contents, not the gzip bytes (codec-independent). This 64-hex string is the content address AND the CAS revision token.
  • packBundle = gzipSync(canonical) (the single S3 blob); unpackBundle = gunzipSync with a 32 MB decompressed cap (zip-bomb guard — it runs on merchant-controlled blobs) and Object.defineProperty on rebuild (so __proto__ becomes real data, not pollution).

Binary assets (assets.ts) ride a manifest at config/assets.json ({ "favicon.ico": { hash, contentType, size } }). The object key is assetHash(bytes) (sha256 hex → dedup + cache-forever). isAssetHash (/^[a-f0-9]{64}$/) is validated because the hash comes from the merchant-editable manifest and is interpolated into an S3 key. The manifest is written as canonical bytes by hand (sorted paths, fixed field order) — JSON.stringify would reorder keys → a spurious new bundle hash → a spurious new version. Allowed content types exclude svg (can script); everything is neutralized to application/octet-stream otherwise, enforced at upload and serve.

Draft save — optimistic concurrency — theme-store.ts ​

saveDraft(ref, files, { expectedRevision }):

  • No expectedRevision (base provisioning) → last-write-wins put.
  • With expectedRevision → CAS under a Postgres row lock: BEGIN; SET LOCAL lock_timeout='5s'; SELECT 1 FROM theme WHERE id=$1 FOR UPDATE; re-read the current draft, and if bundleId(current) !== expectedRevision → DraftConflict (HTTP 409); else put; then COMMIT is .catch(() => {}) — the S3 put is the durable write and the txn changed no rows, so a COMMIT failure (lock release) must not be reported as an error (it would strand the client on a stale revision and spuriously 409 their retry).

saveOverrides(full) — the editor sends the full composed tree; this stores only diffFromBase(base, full), after removal guards (removing the layout / a required template / a protected structural file each throws) and managed-file guards (sw.js, manifest.json can't be in the merchant diff).

theme_file isn't populated here

The schema has a theme_file per-file index, but ThemeStore doesn't populate it ("lands with the editor"). The CAS revision the store actually uses is the content hash of the whole overrides bundle (bundleId), not theme_file.revision.

Compile & publish ​

  • compileTheme minifies each *.css with a string- and comment-aware trimCss (not blind regex — preserves nav :hover, calc(1px + 2px), selector lists), catching per-file errors → fall back to original (one bad stylesheet must never fail a publish). Merchant CSS (customize.css, theme.css) is left byte-for-byte.
  • freezeBundles freezes the immutable source bundle (= overrides, at sourceHash) and the compiled bundle (= composeTheme(base, overrides) compiled, at compiledHash), and promoteBaseCss stitches one sheet and points the manifest at it. An optional freeze-time CAS stops a concurrent merchant save being swept live during rebase.
  • publish verifies the theme exists before freeze (a doomed publish writes no orphan bundles; content-addressed orphans are harmless anyway), then in a txn:
    • Content-addressed idempotency keyed on compiled_hash (not source_hash — a rebase leaves the overrides identical while the output changed): if the latest version's compiled_hash matches, reuse that version, skip the INSERT; else version = latest + 1.
    • makeLive under SELECT … FROM tenants … FOR UPDATE (serializes vs concurrent activate/rollback). Modes: true (always flip), false (cut a version only), if-already-live (flip only if this theme is already live or none is). The purge is enqueued only when the pointer actually moves (an unchanged already-live republish must not spam the outbox).

setLive (activate/switch) and rollback move the pointer the same way, each under the tenant row lock, each enqueuing a purge. Lock ordering is theme-row first, then tenant-row everywhere, to avoid deadlock.

Load for render — loadLiveCompiled ​

One join selects the bundle from the live pointer; loadCompiled uses an in-memory LRU (default max 32, keyed by compiled_hash — always safe because content-addressed = immutable) and Object.freezes the files before caching (shared across requests; render is read-only, editors must clone). Returns the flattened, compiled ThemeFiles.

base ⊕ overrides & rebase — theme-compose.ts, base-propagation.ts ​

  • composeTheme(base, overrides) — copy base except deleted paths (a _deletes control key lists removed base paths), then apply overrides. diffFromBase is the exact inverse (invariant: compose(base, diff(base, full)) === full); it recomputes _deletes, never trusting an incoming one.
  • rebaseToBase re-anchors overrides onto a new base version: CAS the base_version pin; a dirty-draft guard (published source_hash vs bundleId(draft)) refuses if unsaved; makeLive only if this theme is currently live (never hijack a store on a different theme); on makeLive it re-runs the full-document / required-template / section-ref validation; on any error it CASes the base pin back and restores the draft under CAS (a genuine restore failure is a loud AggregateError, never swallowed).
  • Fleet: planBaseRebase previews (per-store shadowed files = overrides ∩ changed-base-files, and blocked reasons: dirty-draft / broken-layout / broken-refs / broken-home); applyBaseRebase runs one store at a time, self-idempotent (a stale retry never republishes byte-identical versions), one failure never aborts the batch.

Object store — data-objects ​

A deliberately tiny ObjectStore seam (put/get/head/delete): immutable content-addressed objects (published versions

  • assets) plus one mutable draft per theme (draft concurrency is the Postgres lock, not here). CdnReadObjectStore serves only content-hash /published/ keys from the CDN and falls back to S3 on any non-2xx or throw — a pure accelerator. The S3 key layout:
stores/<tenantId>/themes/<themeId>/draft/source.gz                    # the one MUTABLE draft (overrides)
stores/<tenantId>/themes/<themeId>/published/source/<sourceHash>.gz    # immutable frozen overrides
stores/<tenantId>/themes/<themeId>/published/compiled/<compiledHash>.gz # immutable render-ready
stores/<tenantId>/themes/<themeId>/assets/<assetHash>                  # raw immutable bytes (NOT gzipped)

Gotchas

  1. __proto__ safety appears twice (bundle pairs + manifest) — object-keyed serialization would silently drop these paths and change the hash.
  2. Idempotency keys on compiled_hash, not source_hash — a rebase leaves the overrides identical while the served output changes.
  3. The manifest must be canonical bytes — any reorder → a new bundle hash → a spurious version.
  4. saveDraft swallows COMMIT failure — the put is the durable write; the txn only holds a lock.
  5. Bundle writes happen outside the publish txn — abort leaves harmless content-addressed orphans; the exists-check is before freeze.
  6. loadBaseSource fails loud (never ?? {}) — a missing base blob would silently serve a blank store.
  7. Compiled LRU values are frozen and shared — render must treat ThemeFiles as read-only.
  8. The 32 MB cap is on decompressed size (unpackBundle runs on merchant blobs).