NNODE LOOP LABruntime observatoryNNEON · Articles in 90 languages
CONNECTING
v24.18.0linux/x64
Microtasks → check → Redis jobs
07

Promises, setImmediate, and BullMQ

Practice Promise chains, async/await, combinators, and setImmediate, then follow a real BullMQ job lifecycle through Redis.

PROCESS IDcurrent server
UPTIMEsince startup
LOOP DELAY P95perf_hooks
UTILIZATIONevent loop
HTTP ROUNDTRIPbrowser → server
LIVE TRACE

Timeline

READY
0 ms
Events will appear hereRun the selected scenario
#TIMESOURCEEVENT
Waiting for an experiment…
CHAPTER 07
DEEP DIVE · FROM BASICS TO CODE

Understanding: Promises, setImmediate, and BullMQ

You can read this chapter before running the experiment. Then return to the live trace and match each concept to a real event.

FIRST, IN PLAIN LANGUAGE

A Promise is not a background thread. Think of it as a box for exactly one future result. The box starts pending and permanently becomes either fulfilled with a value or rejected with an error. then, catch, and finally do not mutate that box: each call creates the next Promise in a chain. BullMQ solves a different problem by storing work in Redis so separate Workers can pick it up now, later, or after a process restart.

TECHNICAL FOUNDATION

The function passed to new Promise(executor) is called synchronously, while then/catch/finally reactions are scheduled as microtasks. An async function always returns a Promise; await pauses only that function and resumes it through a microtask. setImmediate registers a callback for the Node check phase and does not mean “execute immediately.” BullMQ sits above the Event Loop: Queue writes a job to Redis, Worker processes it, and QueueEvents observes global queue events.

Why it mattersThese layers are easy to conflate. A Promise coordinates a result inside a running program, setImmediate yields to a later check phase, and BullMQ persists work across HTTP requests, processes, and machines. Choosing the correct layer makes failures predictable and prevents important work from disappearing during a restart.
WHERE THE WORK RUNS
01YOUR JS CODEfunctions · callbacks
02NODE APIsfs · crypto · timers
03V8 + LIBUVheap · loop · pool
04OPERATING SYSTEMI/O · threads · memory
01 · GLOSSARY

Terms used in this experiment

Understand the words first, then the execution order.

01

Promise

An object for one future outcome: fulfilled with a value or rejected with a reason. Settlement cannot be reversed.

02

Executor

The function passed to new Promise. It runs synchronously and receives resolve/reject; it is mainly useful for adapting callback APIs.

03

Pending / fulfilled / rejected

The three Promise states. Fulfilled and rejected are collectively settled, and a settled Promise never returns to pending.

04

then chain

A chain of new Promises. A returned value feeds the next link, a returned Promise is awaited, and a throw becomes a rejection.

05

Microtask

A high-priority Promise continuation drained after current JavaScript and before the Event Loop advances to another phase.

06

async / await

async wraps the return value in a Promise. await subscribes to a Promise and pauses only the current async function, not Node’s thread.

07

Promise.all

Waits for every input to fulfill and preserves result order. Its first rejection rejects the aggregate but does not cancel remaining work.

08

Promise.allSettled

Waits for every input and returns fulfilled/rejected status objects, useful when one failure must not hide other results.

09

Promise.race / any

race adopts the first settled outcome, including failure. any adopts the first fulfillment or rejects with AggregateError if none fulfill.

10

setImmediate

A Node API for a check-phase callback. It is neither a microtask nor a context-free promise to beat setTimeout.

11

BullMQ

A Redis-backed Node.js job queue that coordinates producers, workers, retries, delays, priorities, and lifecycle events.

12

Redis

BullMQ’s external state store for waiting, active, delayed, completed, and failed jobs. BullMQ is not a durable queue without Redis.

13

Job

A named unit of work with serializable data and options such as attempts, backoff, delay, priority, and removal policies.

14

BullMQ Worker

A job consumer. Its processor may be async, but a regular BullMQ Worker does not automatically move CPU-heavy JavaScript off its process thread.

15

Idempotency

The property that makes replaying the same job safe instead of sending duplicate email, payment, or database effects.

02 · MECHANICS

What happens step by step

Each step maps to an observable runtime state.

  1. 01
    The executor runs now

    new Promise(executor) calls the executor synchronously. Code after resolve still reaches the end of that executor.

  2. 02
    One outcome wins

    The first resolve/reject wins. Resolving with another Promise adopts that Promise’s eventual outcome.

  3. 03
    Reactions become microtasks

    then, catch, and finally do not enter the current stack; they run after the synchronous callback completes.

  4. 04
    Every link returns a Promise

    The next then waits for the previous return. A forgotten return breaks waiting and error propagation.

  5. 05
    await expresses the same rule

    Code before await runs now; continuation resumes after settlement. try/catch handles rejection like .catch().

  6. 06
    A combinator chooses policy

    all, allSettled, race, and any receive already-created values/Promises and aggregate their outcomes differently.

  7. 07
    setImmediate changes the phase

    Its callback enters check. Promise microtasks from the current callback normally run before Event Loop phases continue.

  8. 08
    A producer adds a BullMQ job

    queue.add resolves after Redis stores the job. That confirms enqueueing, not completion of the business operation.

  9. 09
    A Worker moves waiting to active

    An available Worker reserves the job, calls its processor, and turns return/throw into completed/failed.

  10. 10
    Events, retries, and cleanup

    QueueEvents reports lifecycle globally; attempts/backoff retry transient failures; removal policies prevent unbounded Redis growth.

03 · CONTEXT

Where the result needs context

These details explain why similar code can sometimes produce a different trace.

01

Promise does not make synchronous work async

new Promise(() => heavyCpu()) runs heavyCpu immediately on the current thread. CPU-bound work needs a Worker Thread or another process.

02

resolve does not call then immediately

resolve settles or adopts another Promise. A registered then still runs as a microtask after the current stack.

03

finally is transparent—until it is not

A normal return from finally preserves the original outcome, but a throw or rejected Promise from finally replaces it.

04

catch covers the returned chain above it

It receives source rejection, throws from then, and returned rejected Promises. Detached, unreturned work can escape that chain.

05

Promise.all is fail-fast, not cancel-fast

The aggregate rejects early, but network calls, timers, and jobs continue unless their own APIs receive cancellation.

06

A race timeout cancels nothing by itself

The losing Promise keeps running. fetch needs AbortController; BullMQ needs an explicit job cancellation strategy.

07

setImmediate depends on location

Inside one I/O callback, check precedes the next timers pass, so immediate beats a zero timer. There is no universal main-module order.

08

BullMQ Worker is a role, not Worker Thread

It may run in this process, another process, or another machine. An async processor suits I/O, while CPU loops still block their process.

09

The durable state lives in Redis

A pending Promise disappears with its process. A BullMQ job can remain available after its producer HTTP request or process exits.

10

Delivery is not a unique business effect

Retries and stalled-job recovery require idempotency through stable job IDs, database constraints, or processed-operation records.

01
Theory

First understand which parts of Node participate in execution.

02
Simplified code

Then remove instrumentation and focus on the central mechanism.

03
Runtime code

Finally match the model to the code that produces the live trace.

04 · Simplified code

A minimal model without instrumentation

src/demos.js · educational snippetJavaScript
import { setImmediate as waitImmediate } from 'node:timers/promises';
import { Queue, QueueEvents, Worker } from 'bullmq';

const value = await Promise.resolve(2)
  .then(number => number * 3)
  .then(async number => {
    await waitImmediate(); // check phase
    return number + 1;
  })
  .catch(error => {
    console.error(error);
    return 0;
  })
  .finally(() => console.log('cleanup'));

const connection = { host: 'redis', port: 6379 };
const queue = new Queue('examples', { connection });
const events = new QueueEvents('examples', { connection });
const worker = new Worker('examples', async job => {
  return { doubled: job.data.value * 2 };
}, { connection });

await events.waitUntilReady();
const job = await queue.add('double', { value });
console.log(await job.waitUntilFinished(events, 5000));
05 · Runtime code

The complete code executed by the scenario

This is not an alternative example: these are the functions and files used by the Run button.

ACTUAL SOURCE

The source is generated from the real server function. Scenarios using a child process or Worker include every participating file.

src/demos.js
scenario241 lines
import { randomUUID } from 'node:crypto';
import { setImmediate as waitForImmediate } from 'node:timers/promises';
import {
  Queue,
  QueueEvents,
  Worker as BullWorker,
} from 'bullmq';

const sleep = (ms) =>
  new Promise((resolve) => setTimeout(resolve, ms));

function redisConnectionOptions(redisUrl) {
  const parsed = new URL(redisUrl);
  if (parsed.protocol !== 'redis:' && parsed.protocol !== 'rediss:') {
    throw new Error('REDIS_URL должен использовать redis:// или rediss://');
  }

  const database = Number.parseInt(parsed.pathname.slice(1) || '0', 10);
  return {
    host: parsed.hostname,
    port: Number.parseInt(parsed.port || '6379', 10),
    username: parsed.username || undefined,
    password: parsed.password || undefined,
    db: Number.isFinite(database) ? database : 0,
    tls: parsed.protocol === 'rediss:' ? {} : undefined,
    connectTimeout: 1200,
    maxRetriesPerRequest: null,
    enableOfflineQueue: false,
    retryStrategy: () => null,
  };
}

async function runBullMqRoundtrip(emit) {
  if (!process.env.REDIS_URL) {
    emit(
      'bullmq',
      'warning',
      'BullMQ-пример пропущен: задайте REDIS_URL или запустите проект через Docker Compose',
    );
    return;
  }

  const queueName = `node-loop-lab-${process.pid}-${randomUUID().slice(0, 8)}`;
  const connection = redisConnectionOptions(process.env.REDIS_URL);
  let queue;
  let worker;
  let queueEvents;
  const reportedConnectionErrors = new Set();
  const reportConnectionError = (source, error) => {
    const key = `${source}:${error.message}`;
    if (reportedConnectionErrors.has(key)) return;
    reportedConnectionErrors.add(key);
    emit(source, 'error', `${source} error: ${error.message}`);
  };

  try {
    queue = new Queue(queueName, {
      connection,
      defaultJobOptions: {
        attempts: 3,
        backoff: { type: 'exponential', delay: 100 },
        removeOnComplete: false,
        removeOnFail: false,
      },
    });
    queueEvents = new QueueEvents(queueName, { connection });
    worker = new BullWorker(
      queueName,
      async (job) => {
        emit(
          'bullmq-worker',
          'callback',
          `BullMQ Worker взял job ${job.id}: ${job.name}`,
        );
        await sleep(35);
        return {
          thumbnail: `${job.data.file}.webp`,
          processedBy: process.pid,
        };
      },
      { connection, concurrency: 1 },
    );

    queue.on('error', (error) => reportConnectionError('bullmq-queue', error));
    queueEvents.on('error', (error) =>
      reportConnectionError('bullmq-events', error),
    );
    worker.on('error', (error) =>
      reportConnectionError('bullmq-worker', error),
    );

    await Promise.all([
      queue.waitUntilReady(),
      queueEvents.waitUntilReady(),
      worker.waitUntilReady(),
    ]);

    emit(
      'bullmq-producer',
      'schedule',
      'Queue.add сохраняет job в Redis; HTTP-запрос не выполняет job сам',
    );
    const job = await queue.add('make-thumbnail', {
      file: 'avatar.png',
    });
    emit('redis', 'info', `Redis хранит job ${job.id} в состоянии waiting`);

    const result = await job.waitUntilFinished(queueEvents, 5000);
    emit(
      'bullmq-events',
      'result',
      `QueueEvents получил completed для job ${job.id}: ${result.thumbnail}`,
    );
  } catch (error) {
    emit(
      'bullmq',
      'warning',
      `BullMQ недоступен: ${error.message}. Promise-часть сценария уже выполнена.`,
    );
  } finally {
    await Promise.allSettled([worker?.close(), queueEvents?.close()]);
    if (queue) {
      try {
        await queue.obliterate({ force: true });
      } catch {
        // Redis мог стать недоступен во время очистки учебной очереди.
      }
      await queue.close().catch(() => {});
    }
  }
}

async function promisesImmediateBullMq(emit) {
  emit(
    'call-stack',
    'sync',
    'Создаём Promise: executor выполняется синхронно прямо сейчас',
  );

  const basePromise = new Promise((resolve) => {
    emit('promise-executor', 'sync', 'executor вызвал resolve(2)');
    resolve(2);
    emit(
      'promise-executor',
      'sync',
      'Код после resolve ещё выполняется, но повторно изменить outcome уже нельзя',
    );
  });

  const chainedPromise = basePromise
    .then((value) => {
      emit('microtasks', 'callback', `Первый then получил ${value} и вернул ${value * 3}`);
      return value * 3;
    })
    .then(async (value) => {
      await sleep(25);
      emit(
        'timers',
        'callback',
        `Второй then дождался Promise и получил ${value}`,
      );
      return value + 1;
    })
    .finally(() => {
      emit(
        'microtasks',
        'callback',
        'finally выполнился без подмены успешного результата',
      );
    });

  const recoveredPromise = Promise.reject(new Error('учебная ошибка'))
    .catch((error) => {
      emit('microtasks', 'warning', `catch обработал: ${error.message}`);
      return 'fallback';
    });

  const immediatePromise = new Promise((resolve) => {
    setImmediate(() => {
      emit(
        'check',
        'callback',
        'setImmediate callback выполняется в check-фазе',
      );
      resolve('immediate');
    });
  });

  emit(
    'call-stack',
    'schedule',
    'then/catch/setImmediate зарегистрированы; текущий стек освобождается',
  );
  const [chainResult, recovered] = await Promise.all([
    chainedPromise,
    recoveredPromise,
    immediatePromise,
  ]);
  emit(
    'result',
    'result',
    `Цепочка завершилась значением ${chainResult}; catch вернул ${recovered}`,
  );

  const aggregateTasks = [
    sleep(45).then(() => 'slow'),
    sleep(10).then(() => 'fast'),
    sleep(25).then(() => 'middle'),
  ];
  const allResult = await Promise.all(aggregateTasks);
  emit(
    'promise-all',
    'result',
    `Promise.all сохранил входной порядок: ${allResult.join(', ')}`,
  );

  const settled = await Promise.allSettled([
    Promise.resolve('ok'),
    Promise.reject(new Error('expected failure')),
  ]);
  emit(
    'promise-all-settled',
    'result',
    `Promise.allSettled вернул статусы: ${settled.map((item) => item.status).join(', ')}`,
  );

  const raceWinner = await Promise.race([
    sleep(30).then(() => 'timer 30ms'),
    waitForImmediate().then(() => 'setImmediate'),
  ]);
  emit('check', 'result', `Promise.race: первым завершился ${raceWinner}`);

  await waitForImmediate();
  emit(
    'check',
    'callback',
    'await setImmediate() из node:timers/promises продолжил функцию в check-фазе',
  );

  await runBullMqRoundtrip(emit);
}

The emit(...) calls become live-trace rows. await and Promise keep the HTTP stream open until the scenario completes.

06 · RECIPES

Practical patterns worth keeping nearby

Compare the goal, code, and caveats instead of memorizing syntax without a model.

01

01 · Adapt a callback to Promise

Use the constructor at an old callback boundary; an already promise-based API does not need another constructor.

import { readFile } from 'node:fs';

function readText(path) {
  return new Promise((resolve, reject) => {
    readFile(path, 'utf8', (error, text) => {
      if (error) return reject(error);
      resolve(text);
    });
  });
}
  • The executor runs synchronously.
  • resolve/reject run later from the callback.
  • fs/promises already exists; this wrapper is educational.
02

02 · A then chain and its required return

Each return defines the next input and connects errors into one chain.

getUser(42)
  .then(user => {
    return getOrders(user.id);
  })
  .then(orders => orders.filter(order => order.paid))
  .then(paid => console.log(paid))
  .catch(error => console.error(error))
  .finally(() => console.log('finished'));
  • The first then can be shortened to `.then(user => getOrders(user.id))`.
  • A throw from any returned link reaches catch.
  • finally receives neither paid nor error as an argument.
03

03 · The same flow with async/await

async/await changes the syntax while remaining Promise and microtask machinery.

async function printPaidOrders(userId) {
  try {
    const user = await getUser(userId);
    const orders = await getOrders(user.id);
    const paid = orders.filter(order => order.paid);
    console.log(paid);
    return paid;
  } catch (error) {
    console.error(error);
    throw error;
  } finally {
    console.log('finished');
  }
}
  • The returned paid value becomes fulfillment.
  • Rethrowing keeps the caller aware of failure.
  • Use sequential awaits only for real dependencies.
04

04 · Choose a Promise combinator

Start operations first, then choose the policy for waiting on them.

const tasks = urls.map(url => fetch(url));

const everyResponse = await Promise.all(tasks);
const everyOutcome = await Promise.allSettled(tasks);
const firstSettled = await Promise.race(tasks);
const firstSuccess = await Promise.any(tasks);
  • all: every fulfillment or the first rejection.
  • allSettled: a complete report without early rejection.
  • race: first success/error; any: first success.
05

05 · Timeout with real fetch cancellation

Promise.race is not enough: AbortController tells the losing operation to stop.

async function fetchWithTimeout(url, timeoutMs = 2000) {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), timeoutMs);

  try {
    return await fetch(url, { signal: controller.signal });
  } finally {
    clearTimeout(timer);
  }
}
  • The caller still has to handle AbortError.
  • Not every Promise API supports cancellation.
  • finally guarantees timer cleanup.
06

06 · setImmediate callback and Promise API

Yield to the check phase without promising a millisecond delay.

import { setImmediate as waitImmediate } from 'node:timers/promises';

setImmediate(() => {
  console.log('check phase callback');
});

await waitImmediate();
console.log('continued in check phase');
  • A current-callback Promise.then normally runs first.
  • This is a Node API, not a browser standard.
  • Use a Worker for CPU work instead of repeatedly yielding a heavy loop.
07

07 · BullMQ producer and Worker

The producer stores a job quickly; a Worker processes it independently from the HTTP controller.

import { Queue, Worker } from 'bullmq';

const connection = { host: '127.0.0.1', port: 6379 };
const queue = new Queue('email', { connection });

await queue.add('welcome', { userId: 42 }, {
  jobId: 'welcome:42',
  attempts: 3,
  backoff: { type: 'exponential', delay: 1000 },
  removeOnComplete: 1000,
  removeOnFail: 5000,
});

const worker = new Worker('email', async job => {
  return sendWelcomeEmail(job.data.userId);
}, { connection, concurrency: 10 });

worker.on('error', error => console.error(error));
  • Queue and Worker can live in separate processes or machines.
  • jobId helps enqueue deduplication but does not replace effect idempotency.
  • concurrency suits I/O; isolate CPU-heavy processors.
08

08 · QueueEvents and one job result

Distinguish local Worker events from queue-wide events.

import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents('email', { connection });
await queueEvents.waitUntilReady();

queueEvents.on('completed', ({ jobId, returnvalue }) => {
  console.log(jobId, returnvalue);
});

const job = await queue.add('welcome', { userId: 42 });
const result = await job.waitUntilFinished(queueEvents, 10_000);

await queueEvents.close();
  • QueueEvents uses a dedicated Redis connection.
  • The wait timeout does not necessarily cancel the job.
  • A long-running service normally reuses one QueueEvents instance.
07 · DO NOT CONFUSE

Common misconceptions

The myth is on the left; the accurate model is on the right.

MYTH

new Promise(async (resolve) => ...) is the normal way to use await.

ACTUALLY

The constructor expects a synchronous executor. An async executor creates a second Promise whose rejection is not automatically connected; move async work into a separate function.

MYTH

then changes the original Promise.

ACTUALLY

then always returns a new Promise. The original outcome remains unchanged.

MYTH

The next then waits even if I forget return.

ACTUALLY

Without return, the chain receives undefined and continues before the detached operation.

MYTH

await blocks the Event Loop until the response arrives.

ACTUALLY

It pauses one async function. The Event Loop can serve other callbacks while the awaited Promise is pending.

MYTH

Promise.all starts functions in parallel.

ACTUALLY

Operations start when the functions are called. Promise.all only aggregates the Promises/values it receives.

MYTH

setImmediate means run right now.

ACTUALLY

It registers a check-phase callback that still waits for a free stack and the proper Event Loop pass.

MYTH

BullMQ is a large Event Loop callback queue.

ACTUALLY

BullMQ is a distributed application queue in Redis; runtime callback/microtask queues belong to one Node process.

MYTH

Successful queue.add means the email has been sent.

ACTUALLY

It means the job was stored. Worker completion or a completed QueueEvents event confirms processing.

08 · SELF-CHECK

Explain it in your own words

If you can explain the answer without quoting documentation, your mental model is starting to take shape.

  1. Why does console.log in the executor appear before console.log in then?
  2. What does the next then receive when the previous one returns nothing?
  3. Why can allSettled be better than all for independent file batches?
  4. Why are setImmediate and BullMQ not the same kind of queue?
  5. How would you make email delivery safe when BullMQ retries a job?