MinIO migration
PutFS was built to replace self-hosted MinIO. Almost the entire feature set maps onto the Linux/ZFS stack and UNIX tooling; the table below is the map.
Assumptions
For OpenAleph and our managed deployments:
- Few buckets, rarely added (each is a
zfs create) - Only operators manage buckets and access
- Simple, rarely changing access policies
- Mostly (not necessarily) WORM data
Tenant and bucket provisioning and access control are part of the deployment. Without that, the absence of an admin UI gets a bit annoying over time.
Feature mapping
Rows with a reference page have PutFS or nginx logic behind them. The rest come from ZFS (or your filesystem of choice) and standard tooling: the data is plain files, so every filesystem feature applies as it is.
| MinIO feature | PutFS | Reference |
|---|---|---|
| Object storage | Plain filesystem | Data model |
| Bucket management | Directories, or ZFS datasets (zfs create tank/putfs/<bucket>) |
Data model |
| IAM policies | nginx map with method/path scopes, or auth_request |
Auth |
| API key management | keys/*.nginx.conf + nginx -s reload |
Auth |
| S3 SigV4 auth | Not implemented; two-header auth or auth_request |
Auth |
| Presigned URLs | nginx secure_link, method-bound, optionally IP-bound |
Presigned URLs |
| Object locking (WORM) | PUTFS_WORM_PATHS; chattr +i per file |
WORM |
| Content verification | PUTFS_CHECKSUM_PATHS: content-addressed keys hashed on upload |
WORM |
| Bucket read-only | zfs set readonly=on, mount -o remount,ro, chattr -R +i |
WORM |
| Legal hold | zfs hold on a snapshot |
WORM |
| Site replication | zrepl (ZFS send/receive), rsync, rclone, DRBD | Replication |
| Read replicas, failover | Load balancer: writes to the primary, reads round-robin | Replication |
| Horizontal scaling | One dataset per node, routed at the load balancer; a dataset never spans nodes | Replication |
| Vertical scaling | zpool add (online, no rebalancing), more RAM for the ARC, NVMe special vdev and SLOG |
ZFS tuning |
| Object versioning | Snapshots (zfs snapshot, btrfs, LVM) or rsync --link-dest. Per dataset at snapshot granularity, not per object; for per-object history use versioned keys |
|
| Point-in-time restore | zfs rollback (btrfs and LVM alike) |
|
| Erasure coding | ZFS mirror / RAIDZ1-3; mdadm RAID 5/6; btrfs RAID. Mirrors beat RAIDZ for small-file random reads | |
| Self-healing, bit-rot | ZFS checksums + a weekly zpool scrub; dm-integrity + mdadm; btrfs scrub |
|
| Encryption at rest | ZFS native encryption per dataset (-o encryption=aes-256-gcm), or LUKS / dm-crypt per volume. The key never touches PutFS |
|
| Storage tiering, caches | ZFS ARC (RAM), L2ARC (SSD), special vdev for metadata and small files (special_small_blocks), SLOG for writes; or OS page cache + bcache / dm-cache |
ZFS tuning |
| Quota management | zfs set quota=100G tank/putfs/<bucket> or filesystem quotas; PutFS answers 507 when full and has no upload size limit of its own |
|
| Object lifecycle, expiry | cron + find <dir> -type f -mtime +90 -delete; zfs destroy for a whole dataset |
|
| Metrics, monitoring | nginx stub_status + nginx-prometheus-exporter; zpool iostat, arcstat, zfs_exporter; a buffered access log as event stream |
ZFS tuning |
| WebDAV, SFTP, desktop mount | nginx dav_methods on the data root (ngx_http_dav_ext_module); OpenSSH internal-sftp chroot; Finder, Cyberduck or rclone over either |
|
| S3 API | Not implemented (prototype removed) | S3 (not implemented) |
Honest gaps
No clean PutFS equivalent, out of scope by design:
| MinIO feature | Why not |
|---|---|
| Active-active multi-site | Needs distributed consensus. Use Garage or Ceph. |
| Full S3 API (multipart, ACLs, ...) | Not implemented. Point S3 clients at Garage, SeaweedFS or MinIO. |
| S3 Select (SQL over objects) | Niche. Use your application layer. |
| Lambda / trigger integration | AWS-specific. Use nginx log pipelines. |
| Admin UI | Use your filesystem, the ZFS CLI, or build your own. |
| Cross-account IAM federation | Use your identity provider via auth_request. |
Alternatives
If the gaps are dealbreakers:
| System | Best for | Trade-off |
|---|---|---|
| SeaweedFS | High-volume distributed storage with filer API | 3 components, own metadata layer |
| Ceph | Large-scale, erasure-coded, multi-tenant | Complex to operate, needs dedicated infra |
| Garage | Geo-distributed, multi-site S3 | Designed for 3+ nodes |
| RustFS | MinIO clone in Rust | Alpha stage, and we find it very sketchy |
| GlusterFS | Distributed POSIX filesystem | Heavy, not designed for object storage |
Migration steps
The data root is a flat filesystem, so ssh into the PutFS server and mc mirror straight onto it.
PutFS has no S3 endpoint. Patch S3-only clients to speak HTTP PUT/GET, or run a real S3 implementation alongside.