Storage Configuration
Where Hadrian stores file bytes — database, local filesystem, or S3-compatible object storage
The [storage] section controls where Hadrian persists the bytes of files. File
metadata (ids, names, sizes, ownership) always lives in the database; this section only
governs where the raw content is written.
There are two independent storage targets:
| Section | Holds |
|---|---|
[storage.files] | Files API objects (/v1/files) — the same resource that backs Knowledge Bases and Responses-API input_file references. |
[storage.container_files] | Container output files — artifacts the agentic shell tool captures under /mnt/data and serves from /v1/containers/{id}/files/{id}/content. |
Both default to the database backend and are configured the same way, so you can keep
small uploaded files inline in the database while routing bulky agent-generated artifacts to
object storage.
Backends
Each target's backend is independent. A common production layout is database for
[storage.files] and s3 for [storage.container_files].
| Backend | Where bytes live | Best for |
|---|---|---|
database | A BYTEA / BLOB column on the metadata row | Small deployments, quick start. Scales poorly for large or numerous files. |
filesystem | Local disk under a configured directory | Single-node deployments with large files. |
s3 | An S3-compatible bucket | Production, multi-node deployments. Requires the s3-storage build feature. |
The database backend stores file bytes directly in the database. This is simple but inflates
database size and backup time, and bottlenecks large reads/writes on the database connection pool.
For container artifacts in particular — which can be large and numerous — prefer filesystem or
s3 in production.
Database backend (default)
No configuration is required. This is the behavior you get out of the box:
[storage.files]
backend = "database"
[storage.container_files]
backend = "database"Filesystem backend
Stores each file as {path}/{file-id}. Suitable for single-node deployments; for multi-node
the directory must be a shared volume (NFS, EFS) reachable by every node.
[storage.container_files]
backend = "filesystem"
[storage.container_files.filesystem]
path = "/var/hadrian/container-files"
create_dir = true # create the directory on startup if missing (default: true)
file_mode = 384 # decimal for 0o600; Unix file permissions for new files
dir_mode = 448 # decimal for 0o700; Unix directory permissions| Setting | Type | Default | Description |
|---|---|---|---|
path | string | (required) | Base directory for stored files. |
create_dir | bool | true | Create path on startup if it does not exist. |
file_mode | int | 384 (0o600) | Unix permission bits for new files. TOML has no octal literal — give the decimal value. |
dir_mode | int | 448 (0o700) | Unix permission bits for the created directory. |
Reads, deletes, and existence checks are confined to path: any stored reference that resolves
outside the configured root (via .., an absolute path, or a symlink) is rejected. A tampered
database row cannot be used to read or delete arbitrary files on the host.
S3 backend
Works with AWS S3 and any S3-compatible service — MinIO, Cloudflare R2, DigitalOcean Spaces,
Backblaze B2, and others. Requires building with the s3-storage feature (included in the
standard and full profiles):
cargo build --features s3-storage[storage.container_files]
backend = "s3"
[storage.container_files.s3]
bucket = "hadrian-container-artifacts"
region = "us-east-1"
key_prefix = "containers/" # objects stored as containers/<file-id>
# Credentials resolve from env (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY)
# or the instance IAM role when omitted.| Setting | Type | Default | Description |
|---|---|---|---|
bucket | string | (required) | Bucket name. |
region | string | (none) | AWS region. Required unless endpoint is set. |
endpoint | string | (none) | Custom endpoint URL for non-AWS services. |
access_key_id | string | (env / IAM) | Access key. Falls back to environment / IAM role when omitted. |
secret_access_key | string | (env / IAM) | Secret key. Falls back to environment / IAM role when omitted. |
force_path_style | bool | false | Use path-style URLs. Required for MinIO and many self-hosted services. |
key_prefix | string | (none) | Prefix prepended to every object key. Useful when sharing a bucket. |
storage_class | string | (none) | Object storage class (e.g. STANDARD_IA, GLACIER). |
server_side_encryption | table | (none) | Server-side encryption — see below. |
Prefer environment variables or an IAM role over inlining access_key_id / secret_access_key in
the config file. When present, these values are redacted from logs.
S3-compatible endpoints
# MinIO
[storage.container_files.s3]
bucket = "hadrian"
endpoint = "http://minio:9000"
force_path_style = true
access_key_id = "${MINIO_ACCESS_KEY}"
secret_access_key = "${MINIO_SECRET_KEY}"
# Cloudflare R2
[storage.container_files.s3]
bucket = "hadrian"
endpoint = "https://<account-id>.r2.cloudflarestorage.com"
region = "auto"Server-side encryption
# SSE-S3 (Amazon-managed keys)
[storage.container_files.s3.server_side_encryption]
type = "aes256"
# SSE-KMS (AWS KMS keys)
[storage.container_files.s3.server_side_encryption]
type = "kms"
key_id = "arn:aws:kms:us-east-1:123456789:key/abc123"How routing works
The backend recorded for a target applies to files written from that point forward. Each
stored file's row records which backend produced it, so reads continue to resolve correctly
even after you switch backends — existing database rows keep serving their inline bytes
while new files land in filesystem / s3. Switching backends does not migrate existing
content; plan a one-off migration if you need every object in the new backend.
Related
- Agents & the shell tool — what produces container files.