Skip to content

WORM & object locking

Write Once Read Many: prevent accidental or malicious overwrites and deletes.

Path regex WORM mode

Application-level WORM enforced by PutFS, scoped by path regex. For filesystems that cannot be made read-only, such as a content-addressed data lake where path = checksum and re-uploads of the same key are expected.

# Write-once paths: comma-separated Python regexes, matched with re.search
PUTFS_WORM_PATHS=^acme-corp/lake/,^acme-corp/archive/

# Response to a PUT on an existing WORM key
PUTFS_WORM_STRICT=false   # default – 204 No Content (lenient)
PUTFS_WORM_STRICT=true    # 409 Conflict (strict)

# DELETE on a WORM key
PUTFS_WORM_ALLOW_DELETE=false   # default – 403 Forbidden
PUTFS_WORM_ALLOW_DELETE=true    # allow

Lenient mode (default). A PUT on an existing WORM key returns 204 No Content without reading the body. Correct by construction for content-addressed paths (same path, same bytes), and sync clients re-upload idempotently with no error handling. Strict mode returns 409 Conflict instead; use it where paths are not content-addressed and a silent skip would hide client bugs.

Mid-upload response can stress some clients

Both modes answer while the client is still streaming the body. That is legal HTTP/1.1 (RFC 9112 §9.6): a client must watch for an early response and stop sending. Older or hand-rolled clients may instead see a write error (BrokenPipeError, ECONNRESET) before they read the response, a spurious failure on a key that is in fact stored (lenient) or rejected (strict). The robust client-side fix is a HEAD before the PUT on content-addressed paths; otherwise test your HTTP library before relying on this in production.

Concurrent re-PUT while a write is in flight

Each PUT streams into a private temp file and is published atomically (rename for plain keys, create-only link for WORM keys), so a key only ever exists on disk as a complete, verified object. A re-PUT racing an in-flight write either wins the create (201) or gets the usual 409/204 once the other writer commits; there is no false success and no lock.

In-flight temp files

A PUT's temp lives beside its target as .putfs-tmp.<random>, so the publish never crosses a filesystem boundary. Temps are filtered from listings and the prefix is reserved (a key using it is 400). A hard crash can orphan a temp; reclaim them out of band, e.g. find /srv/putfs -name '.putfs-tmp.*' -mmin +120 -delete in cron, with the age comfortably above your longest upload.

Delete protection (default on). DELETE on a WORM key returns 403 Forbidden. PUTFS_WORM_ALLOW_DELETE=true leaves only the OS-level protection below.

Content-addressed verification

WORM makes a key write-once; PUTFS_CHECKSUM_PATHS also proves the bytes match a content-addressed URL. The body is hashed in flight and a PUT whose digest disagrees with the one captured from the URL is rejected.

# Comma-separated Python regexes. A match must capture the digest in a named
# group whose name selects the algorithm (sha256, sha1, sha512).
PUTFS_CHECKSUM_PATHS=^acme-corp/lake/(?P<sha256>[0-9a-f]{64})/blob$

Patterns compile at startup, so a malformed regex fails fast. A mismatching body returns 422 Unprocessable Content and only ever reached the private temp, which is unlinked. A pattern that matches without a recognized algorithm group is a 400.

Regex portability

The Go and Rust ports compile the pattern with RE2-family engines, which reject lookarounds and backreferences; such patterns run only under Python and abort the ports at startup. Named groups work in all three.

Checksum paths are automatically write-once

A content-addressed path names its bytes, so an overwrite can never change anything. The checksum patterns are therefore folded into PUTFS_WORM_PATHS automatically: no separate WORM entry, and a re-PUT of an existing key answers 409/204 on the WORM fast path without streaming and hashing a body. Add PUTFS_WORM_PATHS only for prefixes that are not content-addressed.

Per-object immutability

The Linux immutable flag survives rm:

chattr +i /srv/putfs/acme-corp/legal/contract.pdf   # lock
chattr -i /srv/putfs/acme-corp/legal/contract.pdf   # unlock

PutFS DELETE on an immutable file returns 403 Forbidden.

Per-dataset read-only

zfs set readonly=on tank/putfs/acme-corp/archive        # ZFS
mount -o remount,ro /srv/putfs/acme-corp/archive        # any filesystem
chattr -R +i /srv/putfs/acme-corp/archive/              # or recursively immutable

PutFS PUT and DELETE on read-only or immutable paths return 403 Forbidden.

zfs snapshot tank/putfs/acme-corp/legal@hold-2024
zfs hold legal_hold tank/putfs/acme-corp/legal@hold-2024      # prevents destroy
zfs release legal_hold tank/putfs/acme-corp/legal@hold-2024

Further reading