Object storage

Spacelift requires access to object storage buckets to store certain pieces of data. Currently AWS S3, Google Cloud Storage, Azure Blob Storage and S3-compatible object storage (for example Ceph RGW) are supported.

The following table explains each of the buckets that are required. For each bucket, we indicate whether they should have versioning enabled, as well as the retention rules that should be configured. Where the retention is listed as "user configurable", you can adjust the retention to suit your needs. A suggested value is included to give you a reasonable starting point.

Name	Versioning	Retention	Description
Deliveries	No	User configurable (1 day)	Used to store the request and response bodies of outbound webhooks.
Large queue messages	No	2 days	Stores inbound webhook payloads that are too large to be posted to a message queue.
Metadata	No	2 days	Stores the metadata required by workers to process runs.
Modules	Suggested	Infinite	Stores Terraform modules and providers.
Policy inputs	No	User configurable (7 days)	Stores sampled policy evaluations.
Run logs	Suggested	User configurable (60 days)	Stores the content of run logs.
Run observability	Suggested	User configurable (60 days)	Stores run observability artifacts: OpenTelemetry traces, CPU profiles, and structured JSON logs.
States	Required	Infinite	Stores OpenTofu/Terraform state files when using the built-in Spacelift state server.
Uploads	Suggested	1 day	Used to temporarily upload files from the frontend.
User uploaded workspaces	Suggested	1 day	Used to store workspaces uploaded from user machines as part of the local preview functionality.
Workspace	Suggested	7 days	Used to temporarily store the workspaces of runs that go into an unconfirmed state.

Configuration

The following environment variables can be used to configure object storage:

Environment variable	Required	Description
OBJECT_STORAGE_TYPE	No	Can be set to aws (AWS S3), gcp (Google Cloud Storage), azure (Azure Blob Storage), s3compatible (S3-compatible storage). Defaults to aws.
OBJECT_STORAGE_AZURE_ACCOUNT_URL	Yes if azure	The URL of the Azure storage account e.g. https://{account}.blob.core.windows.net
OBJECT_STORAGE_S3COMPATIBLE_ENDPOINT	Yes if s3compatible	The S3-compatible storage endpoint as host:port, without a scheme (e.g., storage.example.com:9000).
OBJECT_STORAGE_S3COMPATIBLE_ACCESS_KEY_ID	Yes if s3compatible	The access key ID for S3-compatible storage authentication.
OBJECT_STORAGE_S3COMPATIBLE_SECRET_ACCESS_KEY	Yes if s3compatible	The secret access key for S3-compatible storage authentication.
OBJECT_STORAGE_S3COMPATIBLE_USE_SSL	No	Whether to use SSL/TLS for S3-compatible storage connections. Defaults to false.
OBJECT_STORAGE_S3COMPATIBLE_ALLOW_INSECURE	No	Whether to allow insecure SSL connections to S3-compatible storage. Defaults to false.
OBJECT_STORAGE_S3COMPATIBLE_REGION	No	The region requests are signed for. Only needed if your store enforces a specific region. Defaults to us-east-1.
OBJECT_STORAGE_BUCKET_DELIVERIES	Yes	Bucket where webhook delivery traces are stored.
OBJECT_STORAGE_BUCKET_LARGE_QUEUE_MESSAGES	Yes	Bucket where message payloads too large for storing on message queues are stored.
OBJECT_STORAGE_BUCKET_METADATA	Yes	Bucket used to store metadata needed for workers to execute runs.
OBJECT_STORAGE_BUCKET_MODULES	Yes	Bucket where OpenTofu/Terraform module source is stored.
OBJECT_STORAGE_BUCKET_POLICY_INPUTS	Yes	Bucket where policy evaluation samples are stored.
OBJECT_STORAGE_BUCKET_RUN_LOGS	Yes	Bucket where run logs are stored.
OBJECT_STORAGE_BUCKET_RUN_OBSERVABILITY	No	Bucket where run observability artifacts are stored. Falls back to the run logs bucket when unset.
OBJECT_STORAGE_BUCKET_STATES	Yes	Bucket used to store stack state files.
OBJECT_STORAGE_BUCKET_UPLOADS	Yes	Bucket used to temporarily store files uploaded from the frontend.
OBJECT_STORAGE_BUCKET_UPLOADS_URL	Yes	The URL of the uploads bucket. This is used to generate a Content Security Policy (CSP) to allow the frontend to upload to this bucket.
OBJECT_STORAGE_BUCKET_USER_UPLOADED_WORKSPACES	Yes	Bucket where workspaces uploaded as part of local preview functionality are stored temporarily.
OBJECT_STORAGE_BUCKET_WORKSPACE	Yes	Bucket where run workspaces are stored.

Deprecated MinIO names

The s3compatible type used to be called minio, and its variables used the OBJECT_STORAGE_MINIO_ prefix (for example OBJECT_STORAGE_MINIO_ENDPOINT). The old names still work but are deprecated, and v8.0.0 removes them. If both the old and the new variable are set, the new one wins.

Bucket name formatting

When configuring bucket names, ensure they are exact and do not contain any special characters, trailing or leading spaces, or trailing slashes. Incorrect formatting (e.g., my-bucket/ instead of my-bucket) can cause 400 Bad Request errors and installation failures. Always verify that bucket names are clean strings without prefixes, suffixes, or whitespace.

Access requirements

None of our buckets need public access - they just need access from the Spacelift backend services. For certain situations where access to buckets is required from outside the backend services, for example when uploading state files from the frontend, or when workers upload run logs, we rely on pre-signed URLs that are only valid for a certain period of time.

S3-compatible storage requirements

Any store that implements the S3 API should work, as long as it supports the following:

• Path-style requests: Spacelift addresses buckets as https://<endpoint>/<bucket>/<key>, not through bucket subdomains.
• Region: Spacelift signs requests for us-east-1 by default. Most stores accept any region, but if yours enforces a specific one, set OBJECT_STORAGE_S3COMPATIBLE_REGION to match it.
• Object versioning on the states bucket, including listing object versions and reading or copying a specific version. State history and rollback depend on it. Not every S3-compatible store has it: for example, Garage doesn't support object versioning.
• Multipart uploads and pre-signed URLs (GET and PUT, signed with SigV4).
• Custom metadata headers (x-amz-meta-*), including names with underscores. If a reverse proxy sits in front of your store, make sure it passes these headers through.
• Lifecycle rules for the buckets with a limited retention in the table above. Spacelift doesn't delete expired objects from those buckets itself, so if your store can't expire objects, you need your own cleanup job.

Pre-signed URLs point at the same endpoint the backend uses, so your browsers and workers need to be able to reach the store at that address too.

Authentication

AWS S3Azure Blob StorageGoogle Cloud StorageS3-compatible

When using AWS S3, Spacelift uses the default AWS SDK credential provider chain, which automatically finds credentials in the following order: environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), shared credential files and IAM roles for compute resources like EC2, ECS and EKS.

When using Azure Blob Storage, Spacelift uses the DefaultAzureCredential authentication chain. This automatically supports multiple authentication methods including environment variables, Workload Identity (recommended for AKS), and Managed Identity.

When using Google Cloud Storage, Spacelift uses Application Default Credentials (ADC), which automatically finds credentials from multiple sources including: GOOGLE_APPLICATION_CREDENTIALS environment variable, attached service accounts for compute resources, and Workload Identity Federation for containerized environments like GKE.

When using S3-compatible storage, Spacelift requires explicit configuration through environment variables: OBJECT_STORAGE_S3COMPATIBLE_ENDPOINT for the server endpoint, OBJECT_STORAGE_S3COMPATIBLE_ACCESS_KEY_ID for the access key, and OBJECT_STORAGE_S3COMPATIBLE_SECRET_ACCESS_KEY for the secret key.

Uploads bucket CORS configuration

The uploads bucket is used to allow users to upload certain items like Terraform state files from the frontend. To allow this to work, the bucket needs to be configured with the frontend URL as an allowed origin. The following examples show how to configure this using Terraform assuming you were hosting Spacelift at spacelift.myorg.com:

AWS S3Google Cloud StorageAzure Blob Storage

resource "aws_s3_bucket_cors_configuration" "spacelift-uploads" {
    bucket = aws_s3_bucket.spacelift-uploads.id

    cors_rule {
        allowed_headers = ["*"]
        allowed_methods = ["PUT", "POST"]
        allowed_origins = ["https://spacelift.myorg.com"]
    }
}

resource "google_storage_bucket" "spacelift-uploads" {
    name = # bucket name
    location = # location

    public_access_prevention = "enforced"

    cors {
        origin = ["https://spacelift.myorg.com"]
        method = ["PUT", "POST"]
        response_header = ["*"]
        max_age_seconds = 3600
    }

    ... # more configuration options
}

resource "azurerm_storage_account" "spacelift_storage_account" {
  # Some of the configuration options are omitted for brevity.
  blob_properties {
    versioning_enabled = true
    cors_rule {
      allowed_headers    = ["*"]
      allowed_methods    = ["PUT", "POST"]
      allowed_origins    = ["https://spacelift.myorg.com"]
      exposed_headers    = ["*"]
      max_age_in_seconds = 3600
    }
  }
}

Note that the platform-specific Terraform modules (terraform-azure-spacelift-selfhosted, terraform-google-spacelift-selfhosted, terraform-aws-spacelift-selfhosted) take care of the CORS configuration.