Message queues»

Spacelift uses a number of message queues to support asynchronous processing. We support two options for message queues: AWS SQS, or a built-in message broker that uses your Postgres database.

For new installations we suggest using the Postgres message broker. The main exception to this is if you want to use AWS IoT Core rather than our built-in MQTT broker. In that case you must use SQS as IoT Core can only deliver messages to SQS queues by design.

Queues»

Spacelift splits its asynchronous processing across the following logical queues.

Queue	Purpose
`async-jobs`	Background operations: run state changes, notifications, drift detection, billing, and similar
`async-jobs.fifo`	Background operations that must be processed in order
`cronjobs`	Scheduled (cron) job triggers
`dlq`	Messages that repeatedly failed processing
`dlq.fifo`	Ordered messages that repeatedly failed processing
`events-inbox`	Internal domain events fanned out to multiple consumers
`iot`	Messages from workers (heartbeats, run progress), bridged from the MQTT broker
`webhooks`	Inbound webhooks from your VCS and other integrations

Configuration»

The following environment variables can be used to configure message queues:

Environment variable	Required	Description
`MESSAGE_QUEUE_TYPE`	No	Can be set to either `sqs` or `postgres`. Defaults to `sqs`.
`MESSAGE_QUEUE_SQS_ASYNC_FIFO_URL`	For `sqs`	The URL of the SQS queue backing the `async-jobs.fifo` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_ASYNC_URL`	For `sqs`	The URL of the SQS queue backing the `async-jobs` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_CRONJOBS_URL`	For `sqs`	The URL of the SQS queue backing the `cronjobs` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_DLQ_FIFO_URL`	For `sqs`	The URL of the SQS queue backing the `dlq.fifo` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_DLQ_URL`	For `sqs`	The URL of the SQS queue backing the `dlq` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_EVENTS_INBOX_URL`	For `sqs`	The URL of the SQS queue backing the `events-inbox` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_IOT_URL`	For `sqs`	The URL of the SQS queue backing the `iot` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.
`MESSAGE_QUEUE_SQS_WEBHOOKS_URL`	For `sqs`	The URL of the SQS queue backing the `webhooks` queue. Required when `MESSAGE_QUEUE_TYPE` is set to `sqs`.

SQS»

When using SQS, create one queue for each of the logical queues below. The name of the AWS resource is up to you — only the URL you pass to the matching MESSAGE_QUEUE_SQS_* variable (see Configuration) matters. Use the following configuration options for each:

Queue	Visibility timeout	Message retention
`async-jobs`	300s	Default
`async-jobs.fifo`	300s	Default
`cronjobs`	300s	3600s
`dlq`	300s	Default
`dlq.fifo`	300s	Default
`events-inbox`	300s	Default
`iot`	45	Default
`webhooks`	600s	Default

In addition, you should use the following configuration options for all the queues:

A max receive count of 3.
The async-jobs.fifo queue should have its redrive policy configured to send messages to the dlq.fifo queue.
All other queues should use the dlq queue.

IoT Core topic rule»

In order to allow messages sent from Spacelift workers to be processed by the backend services, an IoT Core topic rule needs to be created to publish messages sent to certain topics onto your IoT SQS queue.

The rule should look something like this:

resource "aws_iot_topic_rule" "iot-to-sqs" {
  # var.iot_namespace is configurable, but needs to match whatever you use for `MQTT_BROKER_IOTCORE_NAMESPACE` (defaults to `spacelift`).
  name        = var.iot_namespace
  description = "Send all messages published in the ${var.iot_namespace} namespace to the ${aws_sqs_queue.iot.name} queue"
  enabled     = true
  sql         = "SELECT *, Timestamp() as timestamp, topic(3) as worker_pool_ulid, topic(4) as worker_ulid FROM '${var.iot_namespace}/writeonly/#'"
  sql_version = "2016-03-23"

  sqs {
    # queue_url should reference your IoT queue
    queue_url  = aws_sqs_queue.iot.id

    # role_arn should point to a role that is able to perform `sqs:SendMessage` on the IoT SQS queue.
    role_arn   = aws_iam_role.iot.arn
    use_base64 = true
  }
}

Postgres»

The Postgres message broker doesn't require any specific additional configuration other than setting the MESSAGE_QUEUE_TYPE environment variable to postgres.

Monitoring & scaling»

For detailed information on monitoring message queue performance and metrics, see the observability guide.