Background Jobs

RocketFuel includes optional BullMQ integration for background job processing. This guide covers setup, creating jobs, and running workers.

Prerequisites

BullMQ requires Redis. RocketFuel includes a Docker Compose file to run Redis locally.

Starting Redis

Docker Compose
Docker

docker compose up -d

This starts Redis on localhost:6379.

docker run --name redis -p 6379:6379 -d redis

Environment Variables

REDIS_URL="redis://localhost:6379"

Creating Queues

Define queues in src/lib/queue.ts:

import { Queue } from "bullmq";
import { connection } from "./redis";

export const emailQueue = new Queue("email", { connection });
export const reportQueue = new Queue("reports", { connection });

Adding Jobs

Add jobs from anywhere in your application:

import { emailQueue } from "@/lib/queue";

// Add a job
await emailQueue.add("send-welcome", {
  to: "user@example.com",
  subject: "Welcome!",
});

// Add with options
await emailQueue.add(
  "send-reminder",
  { userId: "123" },
  {
    delay: 60000, // 1 minute delay
    attempts: 3, // Retry up to 3 times
    backoff: {
      type: "exponential",
      delay: 1000,
    },
  }
);

Creating Workers

Workers process jobs from queues. Create in src/scripts/worker.ts:

import { Worker } from "bullmq";
import { connection } from "@/lib/redis";

const emailWorker = new Worker(
  "email",
  async (job) => {
    console.log(`Processing ${job.name}:`, job.data);

    switch (job.name) {
      case "send-welcome":
        // Send welcome email
        break;
      case "send-reminder":
        // Send reminder email
        break;
    }
  },
  { connection }
);

emailWorker.on("completed", (job) => {
  console.log(`Job ${job.id} completed`);
});

emailWorker.on("failed", (job, err) => {
  console.error(`Job ${job?.id} failed:`, err);
});

Running Workers

Add a script to package.json

{
  "scripts": {
    "worker": "tsx src/scripts/worker.ts"
  }
}

Start the worker
Terminal window
```
npm run worker
```

Scheduled Jobs

Run jobs on a schedule using BullMQ’s repeat option:

await emailQueue.add(
  "daily-digest",
  {},
  {
    repeat: {
      pattern: "0 9 * * *", // Every day at 9 AM
    },
  }
);

Monitoring

Use Bull Board or Arena to monitor queues visually.