Skip to content

Storage Providers Management ​

Overview ​

ODP supports multiple storage backends for file storage. Storage providers are configured via the admin API and stored in odp_extension_settings (with extension_key = 'system_storage_provider'). After any change, the StorageManager is re-initialized to pick up the new configuration.

Each provider has two lifecycle flags, stored in its JSON config: isActive (accepts new uploads) and isDefault (the storage used when an upload names none). See Active & Default Providers.

Supported Drivers ​

typeDescription
localLocal filesystem storage
s3Amazon S3 or S3-compatible (e.g., MinIO, Backblaze B2). Optionally scan-gated (malware scan)

A built-in local driver is always available, even when it is not configured — see Implicit local driver.


Sensitive Fields ​

The following storage fields are encrypted at rest using the server SECRET (STORAGE_SENSITIVE_FIELDS in service/api/src/utils/encryption.ts):

  • accessKeyId
  • secretAccessKey

GET /system/storage-providers returns them masked as "********". On PATCH, sending the masked value keeps the stored secret.


Active & Default Providers ​

Flags ​

Field (in value)TypeLegacy config (field missing)Description
isActivebooleanactivefalse = closed to new uploads. Files already stored on it stay readable and deletable
isDefaultbooleannot defaultThe provider used when an upload does not name a storage

Rules (readProviderFlags in service/api/src/storage/manager.ts, assertValidFlags in service/api/src/routes/storage-providers.ts):

  • Both flags are optional and, when sent, must be booleans — otherwise 400 INVALID_PAYLOAD ("isActive" must be a boolean).
  • An inactive provider cannot be the default. Sending isDefault: true together with isActive: false is rejected with 400 INVALID_PAYLOAD (An inactive storage provider cannot be the default). The check looks at the value sent in the request.
  • Only one default. When a provider is saved (create or update) with isDefault: true, the server sets isDefault: false on every other provider.
  • If several stored configs still carry isDefault: true (e.g. edited directly in the DB), the first one by id wins and a warning is logged.

Default resolution ​

When an upload names no storage, StorageManager.getDefaultName() picks, in order:

  1. the provider flagged isDefault, if it is registered and active;
  2. otherwise local, if active;
  3. otherwise the first active driver.

A provider flagged isDefault whose driver failed to initialize (bad config, missing credentials) is not registered, so resolution falls through to local.

Inactive providers ​

An inactive provider is still registered, so files already on it keep working (serve, transform, delete). It is only closed to new uploads:

PathBehaviour when the named storage is inactive or unknown
POST /files (multipart, storage field) and POST /files/import400 INVALID_PAYLOAD — Storage "<name>" is inactive and does not accept new uploads / Storage "<name>" does not exist
POST /tus (storage in Upload-Metadata)400 with the same message as body
GET /files/storage-locationsthe provider is left out

Replacing the content of an existing file (PATCH /files/:id with a file) writes to the storage the file already lives on.

Implicit local driver ​

  • With no provider rows at all, the manager registers a single local driver rooted at STORAGE_LOCAL_ROOT.
  • With provider rows, a local driver (rooted at STORAGE_LOCAL_ROOT) is still registered next to them, unless a provider is itself named local. It is always active and is the fallback default.

Uploads to `local` are not malware-scanned

Only a scan-gated S3 provider (malwareScan: true + rawBucket) puts uploads behind the malware scan gate. The implicit local driver does not. Uploads that name no storage go to the default provider, and when no provider is flagged isDefault that default is local.

Before deploying to an environment whose only configured provider is a scan-gated S3 provider, set that provider as Default (isDefault: true). Otherwise uploads fall back to local and bypass the malware scan.

Upload locations for clients ​

GET /files/storage-locations (any authenticated user) lists the storages that accept new uploads — active drivers only, the default first:

json
{ "data": ["s3-media", "local"] }

With ?detailed=true:

json
{
  "data": [
    { "name": "s3-media", "isDefault": true },
    { "name": "local", "isDefault": false }
  ]
}

Endpoints ​

All endpoints below require Admin access.

GET /system/storage-providers ​

List all configured storage providers (sensitive fields masked). The implicit local driver is not listed — it has no row.

Response:

json
{
  "data": [
    {
      "id": 1,
      "name": "disk",
      "extension_key": "system_storage_provider",
      "value": {
        "type": "local",
        "root": "/var/uploads",
        "isActive": true,
        "isDefault": false
      }
    },
    {
      "id": 2,
      "name": "s3-media",
      "extension_key": "system_storage_provider",
      "value": {
        "type": "s3",
        "bucket": "my-media-bucket",
        "region": "us-east-1",
        "accessKeyId": "********",
        "secretAccessKey": "********",
        "isActive": true,
        "isDefault": true
      }
    }
  ]
}

GET /system/storage-providers/:name ​

Get a specific provider by name.

URL Parameters:

  • name — Provider name

Response: { "data": { ...provider } } — a single provider object.

Error: 404 NOT_FOUND if provider does not exist.


POST /system/storage-providers ​

Create a new storage provider.

Request Body:

json
{
  "name": "s3-archive",
  "value": {
    "type": "s3",
    "bucket": "my-archive-bucket",
    "region": "eu-west-1",
    "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
    "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "isActive": true,
    "isDefault": true
  }
}
FieldTypeRequiredDescription
namestringYesUnique provider name (used in odp_files.storage)
valueobjectYesDriver configuration — see Driver Configurations

Response: 201 Created

json
{
  "data": { "id": 3 }
}

Errors: 400 INVALID_PAYLOAD when name/value is missing or the flags are invalid.

Side Effects: with isDefault: true, every other provider's isDefault is cleared; then storageManager.initialize() reloads all providers.


POST /system/storage-providers/test ​

Test a provider config without saving or registering it. Used by the admin form's "Test connection" button.

Request Body:

json
{
  "id": 2,
  "value": {
    "type": "s3",
    "bucket": "my-media-bucket",
    "region": "us-east-1",
    "accessKeyId": "********",
    "secretAccessKey": "********"
  }
}
FieldTypeRequiredDescription
valueobjectYesThe config to test (may be unsaved form values)
idnumber | stringNoExisting provider id. Its stored (decrypted) config is the base: value is merged over it, and accessKeyId / secretAccessKey sent as "********", "" or omitted are replaced by the stored secrets. 404 NOT_FOUND if the id does not exist

What is probed (service/api/src/storage/test-connection.ts):

typeChecks
s3Requires accessKeyId, secretAccessKey, bucket (else a config step failure MissingConfig). Writes a probe object .odp-connection-test/<uuid>.txt to bucket then runs put → head → delete (with aws:kms SSE when kmsKeyId is set, like real uploads). When malwareScan: true and rawBucket is set, the same probe runs on rawBucket too. The S3 client makes a single attempt (no retries)
localResolves root (default ./uploads), then mkdir → write → stat → unlink a probe file .odp-connection-test-<uuid>.txt
otherconfig step failure UnknownType

Each step has a 5 s timeout (CONNECTION_TEST_TIMEOUT_MS); a timeout is reported as code Timeout. The probe object is deleted on a best-effort basis if a later step fails.

Provider failures do not make the request fail: the response is always 200 and the outcome is in data. Credentials are never echoed back — any access key or secret appearing in a provider error message is replaced by ********.

Response: 200 OK

FieldDescription
oktrue when every check passed
messageConnection OK (<locations>) or Connection failed for <location>: <error>
details.typeTested driver type
details.checks[]One per target: target (bucket | rawBucket | root), location (bucket name or resolved path), ok, and on failure step (config, put, head, delete, mkdir, write, stat, unlink), code, error
details.durationMsTotal test duration

Success:

json
{
  "data": {
    "ok": true,
    "message": "Connection OK (my-media-bucket, my-media-raw)",
    "details": {
      "type": "s3",
      "checks": [
        { "target": "bucket", "location": "my-media-bucket", "ok": true },
        { "target": "rawBucket", "location": "my-media-raw", "ok": true }
      ],
      "durationMs": 412
    }
  }
}

Failure:

json
{
  "data": {
    "ok": false,
    "message": "Connection failed for my-media-bucket: Access denied — the credentials lack s3:PutObject/GetObject/DeleteObject on this bucket (or the KMS key) (Access Denied)",
    "details": {
      "type": "s3",
      "checks": [
        {
          "target": "bucket",
          "location": "my-media-bucket",
          "ok": false,
          "step": "put",
          "code": "AccessDenied",
          "error": "Access denied — the credentials lack s3:PutObject/GetObject/DeleteObject on this bucket (or the KMS key) (Access Denied)"
        }
      ],
      "durationMs": 187
    }
  }
}

Error codes that carry a hint in error: AccessDenied, NoSuchBucket, InvalidAccessKeyId, SignatureDoesNotMatch, PermanentRedirect, AuthorizationHeaderMalformed, Timeout, ENOTFOUND, ECONNREFUSED, EACCES, EROFS, ENOTDIR. Other codes are passed through with the raw provider message.

Errors (request-level): 400 INVALID_PAYLOAD when value is missing; 404 NOT_FOUND for an unknown id.


PATCH /system/storage-providers/:id ​

Update a storage provider's configuration.

URL Parameters:

  • id — Provider record ID (integer)

Request Body:

json
{
  "value": {
    "type": "s3",
    "bucket": "updated-bucket",
    "region": "us-west-2",
    "accessKeyId": "********",
    "secretAccessKey": "NEW_SECRET_KEY",
    "isActive": true,
    "isDefault": false
  }
}

value replaces the stored config as a whole — send every field, not just the changed ones. To keep an existing secret, send the masked value "********" for it.

Response:

json
{
  "data": { "id": 3 }
}

Errors:

  • 400 INVALID_PAYLOAD — value missing, or invalid flags (including isDefault: true with isActive: false).
  • 409 CONFLICT — turning malwareScan off while files on this provider are still pending a scan.

Side Effects: with isDefault: true, every other provider's isDefault is cleared; then storageManager.initialize() reloads all providers.

To deactivate the current default, send isActive: false with isDefault: false (or without isDefault). New uploads then go to the next provider in the default resolution order.


DELETE /system/storage-providers/:id ​

Delete a storage provider.

URL Parameters:

  • id — Provider record ID

Response: 204 No Content

Side Effect: storageManager.initialize() reloads providers. Files that referenced this provider by name will fail to serve. To stop new uploads while keeping existing files served, set isActive: false instead.


Driver Configurations ​

Common fields for every driver: type, and the optional isActive / isDefault flags.

Local Driver ​

json
{
  "type": "local",
  "root": "/var/uploads"
}
FieldRequiredDescription
typeYesMust be local
rootNoStorage directory. Defaults to ./uploads; created if it does not exist

S3 Driver ​

json
{
  "type": "s3",
  "bucket": "my-bucket",
  "region": "us-east-1",
  "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
  "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "endpoint": "https://minio.example.com",
  "kmsKeyId": "arn:aws:kms:us-east-1:111122223333:key/example",
  "malwareScan": true,
  "rawBucket": "my-bucket-raw"
}
FieldRequiredDescription
typeYesMust be s3
bucketYesS3 bucket name. For a scan-gated provider this is the CLEANED (serving) bucket
accessKeyIdYesAccess key ID (encrypted at rest)
secretAccessKeyYesSecret access key (encrypted at rest)
regionNoAWS region (default us-east-1)
endpointNoCustom endpoint URL (MinIO, Backblaze, ...). When set, path-style URLs are used
kmsKeyIdNoKMS key for aws:kms server-side encryption
malwareScanNotrue + rawBucket = scan-gated provider: uploads are written to rawBucket and only served from bucket once scanned clean
rawBucketNoRAW (upload) bucket for a scan-gated provider

A provider missing accessKeyId, secretAccessKey or bucket is skipped at initialization (warning logged) and is not available.


Storage Manager ​

The StorageManager (service/api/src/storage/manager.ts) is a singleton that manages driver instances. When initialize() is called:

  1. Reads all system_storage_provider settings from the database (ordered by id)
  2. Decrypts sensitive fields
  3. Reads the isActive / isDefault flags and instantiates drivers (local, S3, or scan-gated S3). Inactive providers are registered too, marked closed to uploads
  4. Registers the implicit local driver (see above)
  5. Swaps the new registry in atomically, so requests never see a half-built one

Drivers are then available via storageManager.getDriver(name); new uploads resolve their target through storageManager.resolveUploadStorage(name).


Data Model ​

Storage providers are stored in odp_extension_settings:

odp_extension_settings (storage providers) ​

ColumnValue/Description
extension_keysystem_storage_provider
nameProvider name (e.g., disk, s3-media)
valueJSON config (including isActive / isDefault) with encrypted sensitive fields

Environment Variables ​

VariableDefaultDescription
STORAGE_LOCAL_ROOT./uploadsRoot of the implicit local driver
SECRET—Used for encrypting sensitive fields

ODP Internal API Documentation