Build Background Jobs That Actually Run (Bull/BullMQ in Node.js)

§The silent failure problem

Your customer signs up. Your app is supposed to send a welcome email. But the email service is slow (2 seconds). So you do this:

app.post('/signup', async (req, res) => {
  const user = await User.create(req.body);
  await sendEmail(user.email, 'Welcome!'); // Blocks request
  res.json({ success: true });
});

Problems: the customer waits 2+ seconds for a response (bad UX). If the email service is down, the entire signup fails. If your app crashes mid-send, the request is lost. There's no retry if the email fails the first time, and no visibility into what happened.

This is why background jobs exist.

§What Bull does, in 60 seconds

Bull is a Redis-backed job queue. Think of it like a todo list that runs in the background (doesn't block requests), retries automatically (email fails? try again in 60 seconds), survives crashes (jobs persist in Redis), and shows you everything that happened (built-in monitoring).

The flow: a request comes in, you enqueue a job, and respond to the user immediately. A Bull worker picks up the job, processes it, and marks it complete or schedules a retry. If it fails, it retries with exponential backoff (10s, 60s, 600s). If it fails 10 times, it moves to a dead-letter queue that you handle manually.

§Setup (5 minutes)

Step 1: install & start Redis

# Local development (Docker)
docker run -d -p 6379:6379 redis:latest

# Production: Use managed Redis (AWS ElastiCache, Upstash, Redis Cloud)

Step 2: install Bull

npm install bull redis
# OR for newer projects
npm install bullmq redis

Step 3: create your first queue

const Queue = require('bull');
const redis = require('redis');

// Create queue (connects to Redis automatically)
const emailQueue = new Queue('emails', {
  redis: {
    host: process.env.REDIS_HOST || 'localhost',
    port: process.env.REDIS_PORT || 6379
  }
});

module.exports = emailQueue;

§Example: an email job that works

Enqueue (in your route handler)

const emailQueue = require('./queues/emailQueue');

app.post('/signup', async (req, res) => {
  const user = await User.create(req.body);

  // Add job to queue (returns immediately)
  await emailQueue.add(
    { email: user.email, name: user.name }, // job data
    {
      attempts: 3,                    // retry 3 times
      backoff: {                      // wait longer each time
        type: 'exponential',
        delay: 2000                   // start at 2 seconds
      },
      removeOnComplete: true          // clean up after success
    }
  );

  res.json({ success: true }); // respond immediately
});

Process (in a separate worker)

const emailQueue = require('./queues/emailQueue');
const { sendEmail } = require('./email');

// Define how to process jobs
emailQueue.process(async (job) => {
  const { email, name } = job.data;

  try {
    console.log(`Sending email to ${email}...`);
    await sendEmail(email, `Welcome, ${name}!`);
    return { success: true }; // marks job complete
  } catch (error) {
    console.error(`Email failed: ${error.message}`);
    throw error; // triggers retry (Bull handles it)
  }
});

// Optional: Listen to job events
emailQueue.on('completed', (job) => {
  console.log(`Email sent to ${job.data.email}`);
});

emailQueue.on('failed', (job, err) => {
  console.error(`Email failed after retries: ${job.data.email}`);
  // Could send to Slack, log to monitoring tool, etc.
});

Run the worker

# In a separate terminal/process
node worker.js

That's it. Your app now handles emails safely, retries on failure, and has full visibility.

§The real scenario: 10K emails/day

Let's say you send 10,000 welcome + promotional emails daily.

Without jobs (the naive approach): requests time out (email service is slow), the email service going down takes your entire app down with it, there's no way to track failures, and you can't retry selectively.

With Bull:

// Daily email campaign
const emailQueue = new Queue('emails', { redis });

// Enqueue 10K jobs at once (returns in milliseconds)
app.post('/campaign/send', async (req, res) => {
  const users = await User.findAll();

  // Add all jobs at once
  const jobs = await emailQueue.addBulk(
    users.map(user => ({
      name: `send-to-${user.id}`,
      data: { email: user.email, campaignId: req.body.campaignId },
      opts: {
        attempts: 3,
        backoff: { type: 'exponential', delay: 2000 }
      }
    }))
  );

  res.json({ queued: jobs.length }); // Done in <1 second
});

// Process: handle 10 emails concurrently
emailQueue.process(10, async (job) => {
  return sendEmail(job.data.email, 'Special offer...');
});

// Monitoring
emailQueue.on('failed', (job, err) => {
  // Log failures for analysis
  console.log(`Failed: ${job.data.email} - ${err.message}`);
  // Send alert if 10%+ are failing
});

§Common patterns

Recurring jobs (every hour)

emailQueue.add(
  { task: 'cleanup-old-files' },
  {
    repeat: {
      every: 3600000 // milliseconds (1 hour)
    }
  }
);

Delayed jobs (send tomorrow)

emailQueue.add(
  { email: user.email },
  {
    delay: 86400000 // milliseconds (1 day)
  }
);

Priority jobs (VIP emails first)

// High priority: 1 (lower number = higher priority)
emailQueue.add(job, { priority: 1 });

// Low priority: 10
emailQueue.add(job, { priority: 10 });

// Process respects priority order
emailQueue.process(async (job) => { /* ... */ });

§Monitoring (Bull Board)

See what's happening in real time:

const { createBullBoard } = require('@bull-board/api');
const { ExpressAdapter } = require('@bull-board/express');

const serverAdapter = new ExpressAdapter();
createBullBoard({
  queues: [emailQueue, smsQueue, pdfQueue],
  serverAdapter
});

app.use('/admin/queues', serverAdapter.getRouter());

Visit http://localhost:3000/admin/queues to see pending jobs (waiting to run), active jobs (currently processing), completed jobs, failed jobs, and the retry timeline.

§Mistakes to avoid

●No retries — job fails once, data loss. Fix: set attempts: 3+.
●Unlimited retries — a broken job retries forever. Fix: set a max attempts plus a dead-letter queue.
●Process crashes, no recovery — jobs lost if the app crashes. Fix: use PM2/Docker to restart the worker.
●Single worker thread — can't handle volume. Fix: run process(10, job) for concurrency.
●No monitoring — silent failures in production. Fix: use Bull Board or log to a monitoring service.
●Processing the same job twice — double-charges, duplicate data. Fix: use idempotency keys (store processed job IDs).

§Production checklist

●Redis: use a managed service (not your app server)
●Worker: run in a separate process/container
●Monitoring: Bull Board or send alerts to Slack
●Logging: log every job completion + failure
●Retries: set sensible backoff (exponential, not instant)
●Dead-letter: handle jobs that fail all retries
●Scaling: run multiple workers on multiple servers if needed

§Your competitive edge

Founders using job queues don't lose customer data (retries work), scale to 10K events/day without breaking, know exactly what happened (monitoring), and ship faster (background processing means simpler code).

Shipped it but want a second pair of eyes on your copy, DNS, or email deliverability? bitroot.club does a $0 launch review for anyone who followed this guide. →