Skip to content

CLI

The PutFS client command-line interface works similar like awscli or mc (the old minio client). Sync (bulk transfers) is multi-threaded by default.

pip install putfs[client]

Authentication

Via flags or environment variables:

# Environment (recommended)
export PUTFS_API_KEY=my-key
export PUTFS_API_SECRET=my-secret
export PUTFS_ENDPOINT_URL=putfs://localhost:8000
export PUTFS_HTTPS=0

# Or per-command
putfs --key my-key --secret my-secret <command>

Commands

sync

Sync files between any two stores. Uses threaded parallel transfers with streaming (no buffering).

# Local directory → PutFS
putfs sync /data/exports putfs://localhost:8000/acme/exports

# PutFS → local directory
putfs sync putfs://localhost:8000/acme/exports /backup/exports

# Between two PutFS instances
putfs sync putfs://primary:8000/acme putfs://secondary:8000/acme

# Force re-upload everything (skip diff checks)
putfs sync --overwrite /data/exports putfs://localhost:8000/acme/exports

# Delete extra files on target
putfs sync --delete /data/exports putfs://localhost:8000/acme/exports

# Control parallelism (default: min(CPU count, 4))
putfs sync -w 16 /data/exports putfs://localhost:8000/acme/exports

# Retry budget per key on transient faults (default: 5)
putfs sync --retries 10 /data/exports putfs://localhost:8000/acme/exports

Without --overwrite, sync compares mtime (with size as fallback) and skips unchanged files. Without --delete, only the source is listed – the target is not scanned upfront.

Transient faults – read timeouts, dropped connections, 429 and transient 5xx – are retried per key with exponential backoff and jitter. The budget is --retries (default PUTFS_SYNC_RETRIES, 5) and the base delay is PUTFS_SYNC_RETRY_BACKOFF (default 1 second, capped at 60). Permanent answers are not retried: 403, 404, 501 and 507 fail the key immediately, and 409 (WORM re-PUT) and 422 (checksum mismatch) are skips, not failures.

A key that exhausts its budget is logged and counted, and the run carries on – a sync over millions of keys must not lose hours of progress to one unreachable file. The final line reports how many keys synced and how many were deleted; if any key failed, the count goes to stderr and the exit code is 1.

ls

# List all keys
putfs ls putfs://localhost:8000/acme/

# Exclude prefix
putfs ls --exclude-prefix temp/ putfs://localhost:8000/acme/

cp

Copy a single file between stores.

# Upload
putfs cp /local/file.pdf putfs://localhost:8000/acme/file.pdf

# Download
putfs cp putfs://localhost:8000/acme/file.pdf /local/file.pdf

# Between stores
putfs cp s3://bucket/file.pdf putfs://localhost:8000/acme/file.pdf

rm

# Delete a single file
putfs rm putfs://localhost:8000/acme/old-file.pdf

# Delete all files under a prefix
putfs rm --recursive putfs://localhost:8000/acme/temp/

URI schemes

URI Store type
putfs://host:port/path PutFS server (tls by default, disable with PUTFS_HTTPS=0)
/local/path Local filesystem
s3://bucket/key S3-compatible (via anystore)
gs://bucket/key Google Cloud Storage (via anystore)
https://example.org/key Http remote that supports file listing (via anystore)

Any URI supported by anystore (via fsspec) works as source or destination.