DLQ redrive

When message processing fails repeatedly (due to database downtimes, API rate-limiting, or software bugs), you don’t want to lose those events. By configuring a Dead Letter Queue (DLQ), Layeron Queue automatically diverts exhausted messages into a separate logical quarantine queue after they reach their maximum retry threshold.

Once you have investigated the failure, corrected the bug in your code, and deployed the fix, you can redrive those messages—moving them from the DLQ back into the main queue for successful processing.

Code Implementation

This example demonstrates declaring a queue with a DLQ, creating an administrative endpoint to monitor dead-lettered messages, and a redrive endpoint to replay them.

import { backend } from "@layeron/core"
import { queue } from "@layeron/modules"

const app = backend()

// 1. Declare the main queue and attach a DLQ
const ordersQueue = queue({
  name: "orders",
  retry: {
    maxAttempts: 3,        // Retry failed orders 3 times before giving up
    backoff: "exponential",
  },
  deadLetter: {
    name: "orders-dlq",    // Logical name of the DLQ resource
    retentionDays: 14,     // Retain failed payloads for 14 days
  },
})

app.use(ordersQueue)

// 2. Active consumer handler that might occasionally fail
ordersQueue.consume(async (message) => {
  const { orderId, amount } = message.payload

  // E.g., if an external payment gateway is down, throwing an error will retry the message.
  // After 3 failed attempts, Layeron moves the message into "orders-dlq" automatically.
  await processPayment(orderId, amount)
})

// 3. Admin Route: Inspect Queue Health & Dead Letters
app.get("/admin/queues/stats", async (request) => {
  // Ensure route is authenticated in production!
  await checkAdminAuth(request)

  // Retrieve standard queue statistics
  const stats = await ordersQueue.stats()

  // Retrieve the 50 most recent dead letter messages
  const dlqData = await ordersQueue.deadLetters({ limit: 50 })

  return Response.json({
    queue: "orders",
    stats: {
      pending: stats.pending,
      leased: stats.leased,
      completed: stats.completed,
      deadLettered: stats.deadLettered, // Total dead-letters routed historically
    },
    deadLetters: dlqData.messages.map((msg) => ({
      id: msg.id,
      originalMessageId: msg.messageId,
      payload: msg.payload,
      attempts: msg.attempts,
      reason: msg.reason,      // The error message thrown by the consumer on last attempt
      failedAt: msg.failedAt,
    })),
  })
})

// 4. Admin Route: Redrive (Replay) Dead Letters
app.post("/admin/queues/redrive", async (request) => {
  await checkAdminAuth(request)

  const body = await request.json() as { limit?: number; messageIds?: string[] }

  // Redrive moves messages out of the DLQ and puts them back into the pending queue.
  const result = await ordersQueue.redrive({
    limit: body.limit ?? 100,
    messageIds: body.messageIds, // Optional: Redrive specific message IDs, or all if omitted
  })

  return Response.json({
    status: "redrive_triggered",
    redrivenCount: result.redriven,
  })
})

// Mock functions for demonstration
async function processPayment(orderId: string, amount: number) {
  // If payment gateway fails, this throws an error...
}
async function checkAdminAuth(request: Request) {}

How It Works

1. Automatic Isolation: When your .consume() handler reaches maxAttempts, Layeron moves the message into the configured DLQ automatically. The message body and headers are preserved exactly as they were sent.
2. Metadata Retention: When a message goes to the DLQ, Layeron attaches the failure reason (the error message string) and a failedAt timestamp. You can query this via .deadLetters() to diagnose exactly why the messages failed without needing to dig through system logs.
3. Programmatic Replay: .redrive() is an atomic operation. It extracts the quarantined messages and enqueues them back into the main orders queue. When redriven, their attempt counter resets, giving the newly deployed, bug-free consumer code fresh attempts to process them successfully.